Regressi Quatiles fr Time Series Data ZONGWU CAI Departmet f Mathematics Uiversity f Nrth Carlia Charltte, NC 28223, USA E-mail: zcai@ucc.edu Abstract I this article we study parametric estimati f regressi quatiles by ivertig a weighted Nadaraya-Wats estimatr (WNW) f cditial distributi fucti, which was rst used by Hall, Wl ad Ya (1999). First, uder sme regularity cditis, we establish the asympttic rmality ad weak csistecy f the WNW cditial distributi estimatr fr -mixig time series at bth budary ad iterir pits, ad we shw that the WNW cditial distributi estimatr t ly preserves the bias, variace, ad mre imprtatly, autmatic gd budary behavir prperties f lcal liear \duble-kerel" estimatrs itrduced by Yu ad Jes (1998), but als has the additial advatage f always beig a distributi itself. Secdly, it is shw that uder sme regularity cditis, the WNW cditial quatile estimatr is weekly csistet ad rmally distributed ad that it iherits all gd prperties frm the WNW cditial distributi estimatr. A simulati study is carried ut t illustrate the perfrmace f the estimates ad a real example is als used t demstrate the methdlgy. Key Wrds: -mixig; asympttic prperties; cditial quatiles; frecastig; lcal liear smthers; Nadaraya-Wats estimatr; predicti iterval; time series aalysis. AMS 1991 subject classicati. Primary 62G07, 62M10; secdary 62G20, 62G30.
1 Itrducti I parametric estimati f regressi fucti, mst ivestigatis are ccered with the regressi fucti m(x), the cditial mea f Y give value x f a predictr. See, fr example, the bks by Hardle (1990), Wahba (1990), Wad ad Jes (1995), ad Fa ad Gijbels (1996) fr gd itrductis ad iterestig applicatis t the geeral subject areas. Hwever, ew isights abut the uderlyig structures ca be gaied by csiderig the ther aspects f the cditial distributi F (y j x) f Y give = x ther tha the mea fucti m(x). Regressi (cditial) quatiles q p (x), 0 < p < 1, f Y give = x, the tpic f this article, cupled with the cditial distributi F (y j x), are key aspects f iferece i varius statistical prblems. Althugh sme idividual quatiles, such as the cditial media, are smetimes f iterest i practice, mre fte e wishes t btai a cllecti f cditial quatiles which ca characterize the etire cditial distributi. Mre imprtatly, ather applicati f cditial quatiles is cstructi f predicti itervals fr the ext value give a small secti f the recet past values i a statiary time series fy 1 ; : : :; Y g, which mtivates this wrk. Estimati f cditial quatiles has gaied particular atteti durig the recet tw decades due t their useful applicatis i varius elds such as medicie, ecmetrics, ad ace as well as the related elds. See, fr example, Hgg (1975), Keker ad Bassett (1978), Cle (1988), Cle ad Gree (1992), Fa ad Gijbels (1996, Sectis 5.5 ad 6.3), Yu ad Jes (1998), ad Hall, Wl ad Ya (1999), t ame just a few. Of particular iterest is the media fucti q 1=2 (x) fr asymmetric distributi, which ca prvide a useful alterative t the rdiary regressi based the mea. Regressi quatiles ca be als useful fr the estimati f predictive itervals. Fr example, i predictig the respse frm a give cvariate = x, estimates f q =2 (x) ad q 1?=2 (x) ca be used t btai a (1? )100% parametric predictive iterval. Hgg (1975) ad Keker ad Bassett (1978) develped ivative prcedures fr iferece abut q p (x) uder the assumpti that ad Y satisfy a liear mdel. I parametric regressi settig, there are several authrs t study the asympttic prperties f kerel ad earest-eighbr type estimati f cditial quatiles, icludig Ste (1977), Lejeue ad Sarda (1988), Trug (1989), Samata (1989), Bhattacharya ad Gagpadhyay (1990), ad Chaudhuri (1991) fr the iid errrs, ad Russas (1969) ad Russas (1991) fr Markvia prcesses, ad Trug ad Ste (1992) ad Bete ad Fraima (1995) fr mixig depedece. It is well kw that kerel type prcedures have serius drawbacks: the asympttic bias ivlves the desig desity s that they ca t be adaptive, ad they have budary eects s that they require budary mdicati. T atteuate these drawbacks, recetly, sme ew methds f estimatig cditial quatiles have bee prpsed. The rst e, a mre direct apprach, by usig the \check" fucti such as rbustied lcal liear smther, was prvided by Fa, Hu ad Trug (1994) ad further exteded by Yu ad Jes (1998). A alterative prcedure is rst t estimate the cditial distributi fucti by usig \duble-kerel" lcal liear techique f Fa, Ya ad Tg (1996) ad the t ivert the cditial distributi estimatr t prduce a estimatr f a cditial quatile, which is called the Yu ad Jes estimatr. See Yu ad Jes (1998) fr details. A detailed cmparis f these tw methds ca be fud i Yu ad Jes (1997, 1998). Accrdig t Yu ad Jes (1998), a particular preferece is the Yu ad Jes type estimatr. 2
As pited ut by Hall, Wl ad Ya (1999), althugh lcal liear estimatrs f the Yu ad Jes type have sme attractive prperties such as budary eects, desig adaptati ad mathematical eciecy (see Fa ad Gijbels, 1996), they have the disadvatage f prducig cditial distributi fucti estimatrs that are t cstraied either t lie betwee 0 ad 1 r t be mte icreasig althugh sme mdicatis i implemetati have bee addressed by Yu ad Jes (1998). I bth these respects, the Nadaraya-Wats (NW) methds are superir, despite their rather large bias ad budary eects. The prperties f psitivity ad mticity are particularly advatageus if the methd f ivertig cditial distributi estimatr is applied t prduce a estimatr f a cditial quatile. T vercme these diculties, Hall, Wl ad Ya (1999) prpsed a weighted versi f the NW estimatr (WNW), which is desiged t pssess the superir prperties f lcal liear methds such as bias reducti ad budary eect, ad t preserve the prperty that the NW estimatr is always a distributi fucti. Hall, Wl ad Ya (1999) discussed the asympttic rmality f the WNW estimatr fr -mixig uder sme strger assumptis (see Remark 1 belw). Hwever, they did t prvide the rigrus theretical justicati ad they did t discuss the budary behavir. The basic techiques are t vel t this article sice the WNW methd was rst used by Hall, Wl ad Ya (1999), but may details ad isights are. The gal f this paper is tw-fld. First, we establish the asympttic rmality ad weak csistecy fr the WNW estimatr f cditial distributi fr -mixig uder a set f weaker cditis at bth budary ad iterir pits. It is therefre shw, t the rst rder, that the WNW methd ejys the same cvergece rates as thse f lcal liear \duble-kerel" prcedure f Yu ad Jes (1998). A imprtat csequece f this study is that the WNW estimatr has desired samplig prperties at bth budary ad iterir pits f the supprt f the desig desity, which seems t be semial. Secdly, we derive the WNW estimatr f cditial quatile by ivertig the WNW cditial distributi estimatr. We shw that the WNW quatile estimatr exists always due t the WNW distributi beig a distributi fucti itself ad that it iherits all advatages frm the WNW estimatr f cditial distributi, as we describe later. Althugh ur iterest i cditial quatile estimati is mtivated by the frecastig frm time series data, we itrduce ur methds i a mre geeral settig (-mixig) which icludes time series mdelig as a special case. Our theretical results are derived uder -mixig assumpti. Fr referece cveiece, we rst itrduce the mixig ceciet. Let F b a be the -algebra geerated by f( t ; Y t )g b t=a. Dee (t) = supfjp (A B)? P (A) P (B)j : A 2 F 0?1; B 2 F 1 t g: It is called the strg mixig ceciet f the statiary prcess f( t ; Y t )g 1?1. If (t)! 0 as t! 1, the prcess is called strgly mixig r -mixig. Amg varius mixig cditis used i literature, -mixig is reasably weak, ad is kw t be fullled fr may time series mdels. Grdetskii (1977) ad Withers (1981) derived the cditis uder which a liear prcess is -mixig. I fact, uder very mild assumptis liear 3
autregressive ad mre geerally biliear time series mdels are -mixig with mixig ceciets decayig expetially. Auestad ad Tjstheim (1990) prvided illumiatig discussis the rle f -mixig (icludig gemetric ergdicity) fr mdel ideticati i liear time series aalysis. Che ad Tsay (1993) shwed that the fuctial autregressive prcess is gemetrically ergdic uder certai cditis. Furthermre, Masry ad Tjstheim (1995, 1997) demstrated that uder sme mild cditis, bth ARCH prcesses ad liear additive autregressive mdels with exgeus variables, which are particularly ppular i ace ad ecmetrics, are statiary ad -mixig. The pla f the paper is as fllws. I Secti 2, we ccetrate the WNW estimatr f cditial distributi. I Secti 3, we discuss the WNW estimatr f cditial quatiles. I bth sectis, the asympttic rmality ad week csistecy f the estimatrs at bth budary ad iterir pits are stated, ad a ad hc estimatr f the asympttic variace is als preseted. I Secti 4, a simulati study is carried ut t illustrate the estimates ad the methdlgy is als applied t a real example. All techical prfs are give i the Appedix. 2 Cditial Distributi Estimatr 2.1 Weighted Nadaraya-Wats estimatr Let p t (x), fr 1 t, dete the weight fuctis f the data 1 ; : : :; ad the desig pit x with the prperty that each p t (x) 0, P p t (x) = 1, ad ( t? x) p t (x) K h (x? t ) = 0: (2.1) Mtivated by the prperty f lcal liear estimatr, the cstrait (2.1) ca be regarded as a discrete mmet cditi (see (3.12) i Fa ad Gijbels, 1996, p.63). Of curse, fp t (x)g satisfyig these cditis are t uiquely deed, ad we specify them by maximizig Q p t (x) subject t the cstraits. The weighted versi f Nadaraya-Wats estimatr f the cditial distributi F (y j x) f Y t give t = x is deed P p bf (y j x) = t (x) K h (x? t ) I(Y t y) P : (2.2) p t (x) K h (x? t ) Nte that 0 b F (y j x) 1 ad it is mte i y. We shw i Therem 1 (belw) that b F (y j x) is rst-rder equivalet t a lcal liear estimatr, which des t ejy either f these prperties, ad mre imprtatly, i Therem 2 that b F (y j x) has autmatic gd behavir at budaries. The atural questi arises regardig hw t chse the weights. The idea is frm the empirical likelihd. Namely, by maximizig P lgfp t (x)g subject t the cstraits P p t (x) = 1 ad (2.1) thrugh the Lagrage multiplier, the fp t (x)g are simplied t p t (x) =?1 f1 + ( t? x) K h (x? t )g?1 ; (2.3) where, a fucti f data ad x, is uiquely deed by (2.1), which esues that P p t(x) = 1. Equivaletly, is chse t maximize L () = 1 h lg f1 + ( t? x) K h (x? t )g : (2.4) 4
I implemetati, the New-Raphs scheme is recmmeded t d the rt f equati L 0 () = 0. 2.2 Samplig prperties I this secti, we establish the weak csistecy with a rate ad the asympttic rmality fr the WNW estimatr F b (y j x) uder -mixig. Nte that Hall, Wl ad Ya (1999) derived the asympttic rmality fr -mixig withut the detailed prfs uder a set f strger cditis. We rst itrduce sme tati. Let g() dete the margial desity f R R t. Dee j = u j K(u) du ad j = u j K 2 (u) du. Let F (i) (y j x) = (@=@x) i F (y j x). Fr expsitial purpses, we csider ly the special case that is a scalar. We w impse the fllwig regularity cditis. B1. Fr xed y ad x, g(x) > 0, 0 < F (y j x) < 1, g() is ctiuus at x, ad F (y j x) has ctiuus secd rder derivative i a eighbrhd f x. B2. The kerel K() is symmetric desity satisfyig C 1 = supu ju K(u)j < 1. B3. The prcess f( t ; Y t )g is -mixig with the mixig ceciet satisfyig (t) = O fr sme > 0. B4. As! 1, h! 0 ad h 3! 1. t?(2+) B5. Let g 1;t (; ) be the jit desity f 1 ad t fr t 2. Assume that jg 1;t (u; v)?g(u) g(v)j M < 1 fr all u ad v. B6. (t) = O? t?3. Remark 1. B2 des t require that K() be cmpactly supprted, which is impsed by Hall, Wl ad Ya (1999). Nte that may well-kw kerels satisfy the assumpti B2 such as the Gaussia desity ad Epaechikv kerel. Als, te that B6 implies B3. I particular, Cditi B6 is weaker tha that i Hall, Wl ad Ya (1999) fr -mixig, which is strger tha -mixig. Furthermre, B4 is weaker tha that i Hall, Wl ad Ya (1999). Hwever, because B4 is always satised by the badwidths f ptimal size (i.e., h?1=5 ), we d t ccer urselves with such reemets. Therem 1. Suppse that Cditis B1-B5 hld. The, as! 1, bf (y j x)? F (y j x) = 1 2 h2 2 F (2) (y j x) + p (h 2 ) + O p ( h)?1=2 I additi, if Cditi B6 hlds true, the p h h bf (y j x)? F (y j x)? B(y j x) + p (h 2 )i D?! N where the bias ad variace are give respectively by : (2.5) 0; 2 (y j x) ; (2.6) B(y j x) = 1 2 h2 2 F (2) (y j x); (2.7) 5
ad 2 (y j x) = 0 F (y j x)[1? F (y j x)]=g(x): (2.8) Remark 2. It may be see frm the therem that rst, the WNW estimatr b F (y j x)! F (y j x) i prbability with a rate, which, f curse, implies that b F (y j x) is csistet. Als, t the rst rder, the WNW methd ejys the same cvergece rates as thse f lcal liear \duble-kerel" prcedure f Yu ad Jes (1998), uder similar regularity cditis. Hwever, Yu ad Jes (1998) treated ly the case f idepedet data. As fr the budary behavir f the WNW estimatr, we er Therem 2 belw. Withut lss f geerality, we csider left budary pit x = c h, 0 < c < 1. Frm Fa, Hu, ad Trug (1994), we take K() t have supprt [?1; 1] ad g() t have supprt [0; 1]. First, we itrduce the fllwig tati. Let L c () = Z c?1 u K(u) du; (2.9) 1? u K(u) ad c be the rt f equati L c () = 0, amely, L c ( c ) = 0. Fr example, c 1:8 fr c = 0:5 ad c 1:1 fr c = 0:6. Figure 1 depicts the slutis f L c () = 0 fr c takig values frm 0:5-0.2 0.0 0.2 0.4 0.6 c=0.5 c=0.6 c=0.7 c=0.8 c=0.9 c=1.0-1 0 1 2 3 Figure 1: Plt f L c () versus fr c takig values frm 0:5 t 1 with icremet 0:1. t 1 with icremet 0:1. Therem 2. Suppse that the cditis f Therem 1 hld. The p h h bf (y j c h)? F (y j c h)? Bc (y) + p (h 2 )i D?! N 0; 2 c (y) ; (2.10) 6
where the bias term is give by ad the variace is with 0 (c) = Z c?1 j = 1 ad 2. Of curse, g(0+) = lim z#0 g(z). B c (y) = h2 0 (c) F (2) (y j 0+) ; 2 1 (c) 2(y) = 2(0) F (y j 0+)[1? F (y j 0+)] c 2 1 (c) g(0+) u Z 2 K(u) c 1? c u K(u) du; ad K j (u) j(c) =?1 f1? c u K(u)g du j These therems reect tw f the majr advatages f the WNW estimatr: (a) depedece f the asympttic bias the desig desity g(), ad ideed its depedece the simple cditial distributi curvature F (2) ( j ); ad (b) autmatic gd behavir at budaries, at least with regard t rders f magitude, withut the eed f budary crrecti. Als, we remark that a similar result (2.10) hlds fr the right budary pit x = 1? c h. If the pit 0 were a iterir pit, the the expressi (2.10) wuld hld with c = 1 ad c = 0. Furthermre, te that sice the prfs f therems 1 ad 2 are similar, we preset ly the detailed prf f Therem 1 ad give the brief utlie f the prf f Therem 2 i the Appedix. The explicit expressi f the asympttic variace f b F (y j x) give i (2.8) gives a mre direct ad simpler way t cstruct the estimate f 2 (y j x) as fllws b 2 (y j x) = 0 b F (y j x) h 1? b F (y j x) i =bg(x); (2.11) where bg(x) is ay csistet desity estimatr f g(x) which might be btaied by lcal liear prcedure. 3 Quatile Estimatr Our iterest here is t estimate the p-th cditial quatile fucti q p (x) f Y t give t = x fr ay 0 < p < 1, deed by q p (x) = iffy 2 < : F (y j x) pg; which is assumed t be uique. As demstrated i Secti 2, the WNW estimatr f cditial distributi fucti F b (y j x) psses advatages: desig adaptati ad budary eects, which are the same as the lcal liear cuterpart, ad mre imprtatly, beig betwee 0 ad 1 ad mte, which lcal liear estimatr des t ejy either f these prperties. We thus dee the WNW type cditial quatile estimatr bq p (x) i priciple t satisfy F b (bqp (x) j x) = p s that bq p (x) = if y 2 < : b F (y j x) p b F?1 (p j x): (3.1) We remark that bq p (x) always exists sice F b (y j x) is betwee 0 ad 1 ad mte i y, ad it ivlves ly e badwidth s that it makes practical implemetati mre appealig. I 7
ctrast, the \duble-kerel" estimatr f Yu ad Jes (1998) has sme diculty f ivertig the cditial distributi estimatr due t lack f mticity, ad it requires chsig tw badwidths althugh the secd badwidth shuld t be very sesitive. Furthermre, we will shw i Therems 3 ad 4 (belw) that the WNW estimatr bq p (x) maitais the afremetied advatages as b F ( j x) des. T this ed, we eed the fllwig additial cditis. C1. Assume that F (y j x) has a cditial desity f(y j x) ad f(y j x) is ctiuus at x. C2. f(q p (x) j x) > 0. Therem 3. Suppse that Cditis B1-B5 hld. The, as! 1, I additi, if Cditis B6 ad C1-C2 are satised, the, bq p (x)! q p (x) i prbability: (3.2) p h h bqp (x)? q p (x)? B p (x) + p (h 2 )i D?! N where the bias ad variace are give respectively by 0; p(x) 2 ; (3.3) B p (x) =? B(q p(x) j x) f(q p (x) j x) ; ad 2 p(x) = 2 (q p (x) j x) f 2 (q p (x) j x) = 0 p[1? p] f 2 (q p (x) j x) g(q p (x)) : A imprtat way f assessig the perfrmace f bq p (x) is by its mea squared errr (MSE). As a applicati f Therem 3, the MSE f bq p (x) is give by MSE (bq p (x)) = h4 4 ( 2 F (2) (q p (x) j x) f(q p (x) j x) ) 2 + 1 h 0 p(1? p) f 2 (q p (x) j x) g(q p (x)) : (3.4) A cmparis f (3.4) with Therem 1 i Yu ad Jes (1998) fr the \duble-kerel" estimatr, (3.1) des t have the extra tw terms frm the vertical smthig \i the y directi". By miimizig MSE i (3.4), therefre, this yields the ptimal badwidth h pt = 2 6 4 2 2 F (2) (q p (x) j x) 2 0 p(1? p)=g(q p (x)) 3 7 5 1=5?1=5 : I the same maer f (2.11), the csistet estimate f 2 p(x) is b 2 p (x) = 0 p[1? p] bf 2 (q p (x) j x) bg(q p (x)) ; where b f(y j x) ca be btaied by usig the lcal liear \duble-kerel" methd f Fa, Ya ad Tg (1996). 8
Similar t Therem 2, we csider the budary behavir f the WNW estimatr bq p (x) i the fllwig therem. The prf f Therem 4 is mitted sice it is similar t that f Therem 3, which may be fud i the Appedix. Therem 4. Suppse that the cditis f Therem 3 hld. The, as! 1, p h h bqp (c h)? q p (c h)? B p;c + p (h )i 2?! D N ; where the bias ad variace are give respectively by 0; 2 p;c ad B p;c =? B c(q p (0+)) f(q p (0+) j 0+) = 2 (c) F (2) (q p (0+) j 0+)?h2 ; 2 1 (c) f(q p (0+) j 0+) 2 p;c = 2 c (q p (0+)) f 2 (q p (0+) j 0+) = 0 (0) p [1? p] 2 1 (c) f 2 (q p (0+) j 0+) g(q p (0+)) : Similarly, we ca derive the MSE f bq p (c h) as fllws MSE (bq p (c h)) = h4 4 ( 2 (c) F (2) (q p (0+) j 0+) 1 (c) f(q p (0+) j 0+) ) 2 + 1 h ad the crrespdig ptimal badwidth is 2 6 0 (0) p(1? p) 2 1 (c) f 2 (q p (0+) j 0+) g(q p (0+)) ; h c;pt = 4 2 2(c) F (2) (q p (0+) j 0+) 0 (c) p(1? p)=g(q p (0+)) 2 3 7 5 1=5?1=5 : 4 Numerical Prperties 4.1 Simulated example We begi with the illustrati with a simulated example f the AR(1) mdel with 505 bservatis Y t = Y t?1 + " t ; where f" t g are iid N(0; 1) ad = 0:6. The rst 500 bservatis are used fr estimati f the cditial distributi ad the last 5 bservatis are left fr cstructi f predictive itervals. Thrughut this secti, the cvariate t is take t be Y t?1 ad the Epaechikv kerel is used. Figures 2(a) ad (b) display the WNW estimates (thi slid ( t =?1:807), dt ( t =?0:733) ad dashed ( t = 0:402) lies) f cditial distributis F (y j x) (three thick lies i Figure 2(a)) f Y t give t = x ad their quatiles q x (p) = x +?1 (p) (three thick lies i Figure 2(b)), where () is the stadard rmal distributi, ad they shw that the perfrmace f the WNW estimate is reasably well. T check the perfrmace i terms f predicti, we cstruct the 95% predictive itervals [bq 0:025 (x); bq 0:975 (x)] fr the last 5 bservatis, summarized i Table 1. All predictive itervals ctai the crrespdig true values. The average legth f the itervals is 3:91, which is 54:7% f the rage f the data. 9
(a) The cditial CDFs fr AR(1) - WNW (b) The cditial quatiles fr Ar(1) - WNW 0.0 0.2 0.4 0.6 0.8 1.0 x= -1.807 x= -0.733 x=0.402-3 -2-1 0 1 2 x= -1.807 x= -0.733 x=0.402-4 -2 0 2 4 0.0 0.2 0.4 0.6 0.8 1.0 Figure 2: (a) Cditial distributis ad their WNW estimates. (b) Cditial quatiles ad their WNW estimates. Thick slid lies - true fuctis, ad thi slid ( t =?1:807), dt ( t =?0:733) ad dashed ( t = 0:402) lies - three dieret values f cvariate t. Table 1: The pst-sample predictive itervals fr AR(1) mdel Observati True Value Predictive Iterval Y 501-0.272 [?2:24; 2:02] Y 502-1.807 [?2:24; 1:64] Y 503-0.733 [?2:56; 0:33] Y 504 0.402 [?2:75; 1:64] Y 505-0.384 [?2:21; 2:02] 4.2 Real example Fially, we csider the Iteratial Airlie Passeger Data fu t ; t = 1; : : :; 144g f Bx ad Jekis (1976, p.531) (mthly ttals (i thusads) f passegers frm Jauary, 1949 t December, 1960). Figure 3(a) shws clearly that the variability icreases as U t icreases (called multiplicative seasality) ad tread. T elimiate the tread ad seasality, we csider trasfrmatis. We rst use the lgarithmic trasfrmati sice fu t g is a series whse stadard deviati icreases liearly with the mea. The trasfrmed series V t = lg(u t ), shw i Figure 3(b), des t display icrease i variability with V t, but shws clearly a liear tread. Secdly, by fllwig the aalysis i Brckwell ad Davis (1991, pp.284-287), we apply the dierece peratr (1? B)(1? B 12 ) t fv t g t btai the ew series Y t = (1? B)(1? B 12 ) V t, shw i Figure 3(c), which des t display ay apparet deviatis frm statiarity. The rst 125 trasfrmed bservatis are used fr estimati ad the last 6 bservatis are left fr predicti. The WNW estimates f cditial distributi f Y t give t = Y t?1 are depicted i Figure 3(d) fr six dieret values f t (0:0086; 0:0014;?0:0459; 0:0120; 0:0318;?0:0501) ad Figure 3(e) gives the WNW estimates f six cditial quatiles. Nw we csider the frecastig fr the last 6 bservatis based bth the WNW ad NW estimatrs which are cmputed by usig the same badwidth fr each case. The 95% predicti itervals are reprted i Table 2, which shws that the WNW methd 10
(a) Time Series Plt fr Airlie Data (b) Time Series Plt fr Lg-trasfrmed Data 100 200 300 400 500 600 5.0 5.5 6.0 6.5 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 (c) Time Series Plt fr Differeced Lg-trasfrmed Data (d) The cditial CDFs fr Airlie Data -0.15-0.10-0.05 0.0 0.05 0.10 0.15 0.0 0.2 0.4 0.6 0.8 1.0 x=0.0086 x=0.0014 x= -0.0459 x=0.0120 x=0.0318 x= -0.0501 0 20 40 60 80 100 120-0.2-0.1 0.0 0.1 0.2 (e) The cditial quatiles fr Airlie Data (f) The estimated cditial quatile surface -0.10-0.05 0.0 0.05 0.10 0.15 x=0.0086 x=0.0014 x= -0.0459 x=0.0120 x=0.0318 x= -0.0501 0.0 0.2 0.4 0.6 0.8 1.0-0.15-0.1-0.05 0 0.05 0.1 0.15 q_x(p) 0.8 0.6 0.4 p 0.2-0.04-0.02 0 0.02 Figure 3: (a) Time series plt f Airlie data. (b) Time series plt f lg-trasfrmed data. (c) Time series plt f twice-diereced lg-trasfrmed data. (d) ad (e) WNW estimates f cditial CDFs ad quatiles fr six dieret values f cvariate t (0:0086; 0:0014;?0:0459; 0:0120; 0:0318;?0:0501). (f) Estimated cditial quatile q x (p) f Y t give Y t?1 = x. 11
Table 2: The pst-sample predictive itervals fr Airlie data Observati True Value P.I. fr WNW P.I. fr NW Y 126 0.001 [?0:101; 0:045] [?0:101; 0:045] Y 127-0.046 [?0:101; 0:055] [?0:101; 0:067] Y 128 0.012 [?0:102; 0:131] [?0:102; 0:131] Y 129 0.032 [?0:101; 0:042] [?0:101; 0:042] Y 130-0.050 [?0:086; 0:044] [?0:086; 0:044] Y 131-0.010 [?0:054; 0:131] [?0:115; 0:141] utperfrms the NW apprach i 2 ut 6 itervals althugh all predictive itervals based bth methds ctai the crrespdig true values. The average legths f the itervals fr WNW ad NW are 0:166 ad 0:179, respectively, which are respective 58:7% ad 63:6% f the rage f the data. Appedix: Prfs Nte that we use the same tati as i Sectis 2 ad 3. Let " t = I(Y t y)? F (y j t ), ad b t (x) = 1? h 2 g 0 (x) 2 2 g(x) (?1 t? x) K h (x? t ) : (A.1) Set t = p h b t (x) " t K h (x? t ), ad J 1 = s h b t (x) " t K h (x? t ) = 1 p t : (A.2) Let C dete a psitive cstat which might take a dieret value at the dieret place. Lemma 1. Uder the assumptis B1-B5, we have Var(J 1 )! 0 F (y j x)[1? F (y j x)] g(x) = 2 (y j x) g 2 (x) 2 (y j x): Prf. It is easy t see that E[ t ] = 0 sice E[" t j t ] = 0, ad i Var(J 1 ) = E + A straightfrward maipulati yields h E 2 t h 2 t t=2 1? t? 1 Cv( 1 ; t ): i = h E h b 2 t (x) " 2 t K 2 h(x? t ) i = 2 (y jx) g 2 (x) + (1): (A.3) Chse d = O(h?1=(1+=2) ) ad decmpse the secd term the right had side f (A.3) it tw terms as fllws = d + t=2 t=2 t=d +1 12 J 11 + J 12 :
Fr J 11, it fllws by Cditi B5 that jcv( 1 ; t )j C h, s that J 11 = O(d h) = (1). Fr J 12, by applyig Therem A.5 i Hall ad Heyde (1980) ad the fact that K h () C h?1, e has which implies that J 12 C h?1 This cmpletes the prf f the lemma. jcv( 1 ; t )j C h?1 (t? 1); (t) C h?1 d?(1+) td Lemma 2. Uder the assumptis B1-B5, we have s that p t (x) = b t (x) f1 + p (1)g. Prf. Dee, fr j 1, =? h 2 g 0 (x) 2 2 g(x) f1 + p(1)g; A j = 1 ( t? x) j K j (x? h t): Usig the same argumets as thse i Lemma 1, we have = (1): A 1 =? 1 2 2 h 2 g 0 (x) + p (h 2 ); A 2 = h 2 g(x) + p (h 2 ); ad A 3 = O p (h 2 ): (A.4) By (6.4) i Che ad Hall (1993), By a Taylr expasi, s that jj ja 1 j A 2? C 1 ja 1 j = O p(h): 0 = A 1? A 2 + 2 A 3? 3 A 4 + ; = A 1 A 2 + 2 A 3 A 2? 3 A 4 A 2 + : Therefre, substitutig (A.4) it the abve equati, we prve the lemma. Prf f Therem 1. It fllws frm Lemma 2 that P bf (y j x)? F (y j x) = [I(Y t y)? F (y j x)] p t (x) K h (x? t ) P p t (x) K h (x? t ) ( h)?1=2 J 1 + J 2 J?1 3 f1 + p (1)g; (A.5) where ad J 2 = [F (y j t )? F (y j x)] p t (x) K h (x? t ); J 3 = 1 b t (x) K h (x? t ): 13
By Cditi B1 ad (2.1) as well as the Taylr expasi, we have J 2 = 1 2 F (2) (y j x) ( t? x) 2 b t (x) K h (x? t ) + p (h 2 ) = B(y j x) g(x) + p (h 2 ) by fllwig the lie f the prf f Lemma 1. Similarly, J 3 = g(x) + p (1): (A.6) Therefre, p h h bf (y j x)? F (y j x)? B(y j x) + p (h 2 )i = g?1 (x) J 1 + p (1): (A.7) This, i cjucti with Lemma 1, implies (2.5). T prve (2.6), it suces t establish the asympttic rmality f J 1 by (A.7). T this ed, we emply the Db's small-blck ad largeblck techique. Namely, partiti f1; : : : ; g it 2 q + 1 subsets with large-blck f size r = r ad small-blck f size s = s. Set q = q = : (A.8) r + s Dee the radm variables, fr 0 j q? 1, The, j = j(r+s)+r?1 i=j(r+s) J 1 = 1 8 < q?1 p : We will shw that, as! 1, i ; j = j=0 q?1 j + j=0 1 E [Q ;2] 2! 0; E [exp(i t Q ;1)]? 1 q?1 j=0 where 2 (y j x) is deed i Lemma 1, ad 1 q?1 j=0 (j+1)(r+s) i=j(r+s)+r i ; ad q =?1 i=q(r+s) j + q 9 = ; 1 p fq ;1 + Q ;2 + Q ;3 g : E 2 j q?1 Y j=0 1 E [Q ;3] 2! 0; E [exp(i t j )]! 0; h E 2 j I j j j " (y j x) p i : (A.9) (A.10)! 2 (y j x); (A.11) i! 0 (A.12) fr every " > 0. (A.9) implies that Q ;2 ad Q ;3 are asympttically egligible i prbability; (A.10) shws that the summads j i Q ;1 are asympttically idepedet; ad (A.11) ad (A.12) are the stadard Lideberg-Feller cditis fr asympttic rmality f Q ;1 fr the idepedet setup. 14
Let us rst establish (A.9). T this eect, we dee the large-blck size r by r = b( h ) 1=2 c ad the small-blck size s = b( h ) 1=2 = lg c. The, as! 1, s =r! 0; ad (=r ) (s )! 0: (A.13) Observe that E [Q ;2 ] 2 = q?1 j=0 Var( j ) + 2 It fllws frm statiarity ad Lemma 1 that F 1 = q Var( 1 ) = q Var 0i<jq?1 j=1 Cv( i ; j ) F 1 + F 2 : 0 1 @ s j A = q s [ 2 (y j x) + (1)]: (A.14) (A.15) Next csider the secd term F 2 the right-had side f (A.14). Let r j = j(r + s ), the r j? r i r fr all j > i, we therefre have jf 2 j 2 s s 0i<jq?1 j 1 =1 j 2 =1?r 2 j 1 =1 j 2 =j 1 +r jcv( j1 ; j2 )j: jcv( r i +r+j 1 ; r j +r+j 2 )j By statiarity ad Lemma 1, e btais jf 2 j 2 j=r +1 jcv( 1 ; j )j = (): (A.16) Hece, by (A.13)-(A.16), we have It fllws frm statiarity, (A.13) ad Lemma 1 that Var [Q ;3 ] = Var 1 E[Q ;2] 2 = O q s?1 + (1) = (1): (A.17) 0 @?q(r+s) j=1 j 1 A = O(? q (r + s )) = (): (A.18) Cmbiig (A.13), (A.17) ad (A.18), we establish (A.9). As fr (A.11), by statiarity, (A.13) ad Lemma 1, it is easily see that 1 q?1 j=0 E 2 j = q E 2 1 = q r 1 r Var 0 @ r j=1 j 1 A! 2 (y j x): I rder t establish (A.10), we make use f Lemma 1.1 i Vlkskii ad Rzav (1959) t btai E [exp(i t Q ;1)]? qy?1 j=0 E [exp(i t j )] 16 (=r ) (s ) 15
tedig t zer by (A.13). It remais t establish (A.12). T this ed, we emply Therem 4.1 i Sha ad Yu (1996) ad Cditi B6 t btai, h E 2 I 1 j 1 j " (y j x) p i C?1=2 E j 1 j 3 It is easy t see that Therefre, by (A.19) ad (A.20), E h 2 1 I j 1 j " (y j x) p C?1=2 r 3=2 1=2 E j 1 j 6 : (A.19) E j 1 j 6 C h?2 : (A.20) i C?1=2 r 3=2 h?1 : Thus, by (A.8) ad the deiti f r, ad usig Cditi B4, we btai 1 q?1 j=0 h E 2 j I j j j " (y j x) p This cmpletes the prf f the therem. I rder t prve Therem 2, we eed the fllwig lemma. Lemma 3. Uder the assumptis B1-B5, we have i C r 1=2?1=2 h?1 C ( h 3 )?1=4! 0: p t (c h) =?1 b c t(c h) f1 + p (1)g; where b c t (x) = [1 + c ( t? x) K h (x? t )]?1 : Prf. Let b = argmax L () s that L 0 b = 0, where L () is deed i (2.4). It suces t shw that b! c i prbability. T this ed, dete by S " the iterval c ". We will shw that fr ay sucietly small ", the prbability teds t e. By the Taylr expasi, sup 2S " L () L ( c ) L ()? L ( c ) = L 0 ( c ) (? c ) + 1 2 L00 ( c ) (? c ) 2 + 1 6 L000 ( ) (? c ) 3 with lyig betwee ad c. It is easy t shw that L 0 ( c ) = p (1); L 00 ( c ) =? 3 (c) g(0+) + p (1); ad L 000 ( ) = O p (1); where 3 (c) = Z c?1 u 2 K 2 (u) [1? c u K(u)] 2 du: 16
This ccludes with prbability tedig t e that whe " is small eugh, fr all 2 S ", which cmpletes the prf f the lemma. L ()? L ( c ) 0; Prf f Therem 2. By replacig b t (x) i t by b c t(c h) ad fllwig the same argumets as thse used i the prf f Therem 1, we ca prve the therem via Lemma 3. Lemma 4. Uder the assumptis B1-B5 ad C1, we have, fr ay! 0, bf (y + j x)? b F (y j x) = f(y j x) + p ( ) + p ( h)?1=2 : Prf. Let The, It is easy t see that Similarly, Therefre, J 4 = 1 b t (x)[i(y t y + )? I(Y t y)] K h (x? t ): bf (y + j x)? b F (y j x) = J4 J?1 3 f1 + p (1)g: (A.21) E(J 4 ) = E [ff (y + j x)? F (y j x)g b t (x) K h (x? t )] = f(y j x) g(x) + ( ): Var(J 4 ) = O ( h)?1 : J 4 = f(y j x) g(x) + p ( ) + p ( h)?1=2 : This, cupled with (A.6) ad (A.21), prves the lemma. We w embark the prf f Therem 3. Prf f Therem 3. First, we prve (3.2). T this ed, by (2.5), we have, fr all x ad y, bf (y j x)! F (y j x) i prbability: It fllws by Therem 1 i Tucker (1967, pp.127-128) that sup jf (y j x)? F (y j x)j! 0 i prbability (A.22) y2< sice F (y j x) is a distributi fucti. The assumpti that q p (x) is uique implies that, fr ay xed x, there is a " = "(x) > 0 such that = (") = mifp? F (q p (x)? " j x); F (q p (x) + " j x)? pg > 0: 17
It is easy t see that the fllwig iequalities hld, P fjbq p (x)? q p (x)j > "g P fjf (bq p (x) j x)? pj > g = P P F b (bq p (x) j x)? F (bq p (x) j x) ( ) F b (y j x)? F (y j x) > sup y > ; which teds t 0 by (A.22). Therefre, (3.2) hlds true. = B p (x) + ( h)?1=2 p (x) u. The, Q (u) P = P h p h?1 p (x) bqp (x)? q p (x)? B p (x) + p (h 2 ) P fbq p (x) q p (x) + g bf (qp (x) + j x) p by Lemma 4. Therefre, h p Q (u) P h?1 (q p (x) j x) P (u) bf (qp (x) j x)?f(q p (x) j x) + p We w prve (3.3). Fr ay u, let bf (qp (x) j x)? p? B(q p (x) j x) i u i?u by Therem 1, where () is the stadard rmal distributi. Therefre, we ish the prf f the therem. Refereces Auestad, B. ad Tjstheim, D. (1990). Ideticati f liear time series: First rder characterizati ad rder determiati. Bimetrika, 77, 669{687. Bhattacharya, P.K. ad Gagpadhyay, A.K. (1990). Kerel ad earest-eighbr estimati f a cditial quatile. The Aals f Statistics, 18, 1400{1415. Bete, G. ad Fraima, R. (1995). Asympttic distributi f smthers based lcal meas ad lcal medias uder depedece. Jural f Multivariate Aalysis, 54, 77{90. Bx, G.E.P. ad Jekis, G.M. (1976). Time Series Aalysis: Frecastig ad Ctrl. Hlde- Day, Sa Fracisc. Brckwell, P.J. ad Davis, R.A. (1991). Time Series: Thery ad Methds, 2d. Spriger-Verlag, New Yrk. Chaudhuri, P. (1991). Nparametric estimates f regressi quatiles ad their lcal Bahadur represetati. The Aals f statistics, 19, 760{777. Che, R. ad Tsay, R. S. (1993). Fuctial-ceciet autregressive mdels. Jural f the America Statistical Assciati, 88, 298{308. Che, S.. ad Hall, P. (1993). Smthed empirical likelihd cdece itervals fr quatiles. The Aals f Statistics, 21, 1166{1181. 18
Cle, T.J. (1988). Fittig smthed cetile curves t referece data. Jural f the Ryal Statistical Sciety, Series A, 151, 385{418. Cle, T.J. ad Gree, P.J. (1992). Smthig referece cetile curves: The LMS methd ad pealized likelihd. Statistics i Medicie, 11, 1305{1319. Fa, J. ad Gijbels, I. (1996). Lcal Plymial Mdelig ad Its Applicatis. Chapma ad Hall, Ld. Fa, J., Hu, T.-C., ad Trug, Y.K. (1994). Rbust parametric fucti estimati. Scadiavia Jural f Statistics, 21, 433{446. Fa, J., Ya, Q. ad Tg, H. (1996). Estimati f cditial desities ad sesitivity measures i liear dyamical systems. Bimetrika, 83, 189{206. Grdetskii, V.V. (1977). O the strg mixig prperty fr liear sequeces. Thery f Prbability ad Its Applicatis, 22, 411{413. Hall, P. ad Heyde, C.C. (1980). Martigale Limit Thery ad its Applicatis. Academic Press, New Yrk. Hall, P., Wl, R.C.L., ad Ya, Q. (1999). Methds fr estimatig a cditial distributi fucti. Jural f the America Statistical Assciati, 94, 154{163. Hardle, W. (1990). Applied Nparametric Regressi. Cambridge Uiversity Press, New Yrk. Hgg, R.V. (1975). Estimates f percetile regressi lies usig salary data. America Statistical Assciati, 70, 56{59. Jural f the Hurvich, C.M., Sim, J.S. ad Tsai, C.-L. (1998). Smthig parameter selecti i parametric regressi usig a imprved Akaike ifrmati criteri. Jural f the Ryal Statistical Sciety, Series B, 60, 271-293. Keker, R. ad Bassett, G.S. (1978). Regressi quatiles. Ecmetrica, 46, 33{50. Lejeue, M.G. ad Sarda, P. (1988). Quatile regressi: a parametric apprach. Cmputatial Statistics ad Data Aalysis, 6, 229{281. Masry, E. ad Tjstheim, D. (1995). Nparametric estimati ad ideticati f liear ARCH time series: Strg cvergece ad asympttic rmality. Ecmetric Thery, 11, 258{289. Masry, E. ad Tjstheim, D. (1997). Additive liear AR time series ad prjecti estimates. Ecmetric Thery, 13, 214{252. Rice, J. (1984), Badwidth selecti fr parametric regressi. The Aals f Statistics, 12, 1215{1230. Russas, G.G. (1969). Nparametric estimati f the trasiti distributi fucti f a Markv prcess. The Aals f Mathematical Statistics, 40, 1386{1400. Russas, G.G. (1991). Estimati f trasiti distributi fucti ad its quatiles i Markv prcesses: Strg csistecy ad asympttic rmality. I Nparametric Fuctial Estimati ad related Tpics (G.G. Russas, ed.), 443{462. 19
Samata, M. (1989). Nparametric estimati f cditial quatiles. Statistics ad Prbability Letters, 7, 407{412. Sha, Q. ad Yu, H. (1996). Weak cvergece fr weighted empirical prcesses f depedet sequeces. The Aals f Prbability, 24, 2098{2127. Ste C.J. (1977). Csistet parametric regressi (with discussi). The Aals f Statistics, 5, 595{645. Trug, Y.K. (1989). Asympttic prperties f kerel estimatrs based lcal media. The Aals f Statistics, 17, 606{617. Trug, Y.K. ad Ste, C.J. (1992). Nparametric fucti estimati ivlvig time series. The Aals f Statistics, 20, 77-97. Tucker, H. (1967). A Graduate Curse i Prbability. Academic Press, New Yrk. Vlkskii, V.A. ad Rzav, Yu.A. (1959). Sme limit therems fr radm fuctis. I. Thery f Prbability ad Its Applicatis, 4, 178{197. Wad, M.P. ad Jes, M.C. (1995). Kerel Smthig. Chapma ad Hall, Ld. Wahba, G. (1990). Splie Mdels fr Observatial Data. SIAM, Philadelphia. Withers, C.S. (1981). Cditis fr liear prcesses t be strg mixig. Zeitschrift fur Wahrscheilichkeitstherie verwadte Gebiete, 57, 477{480. Yu, K. ad Jes, M.C. (1997). A cmparis f lcal cstat ad lcal liear regressi quatile estimati. Cmputatial Statistics ad Data Aalysis, 25, 159{166. Yu, K. ad Jes, M.C. (1998). Lcal liear quatile regressi. Jural f the America Statistical Assciati, 93, 228{237. 20