Effect of Mean on Variance Function Estimation in Nonparametric Regression

Transcription

1 Effect of Mea o Varace Fucto Estmato Noparametrc Regresso Le Wag, Lawrece D. Brow, T. Toy Ca ad Mchael Leve Abstract Varace fucto estmato oparametrc regresso s cosdered ad the mmax rate of covergece s derved. We are partcularly terested the effect of the ukow mea o the estmato of the varace fucto. Our results dcate that, cotrary to the commo practce, t s ofte ot desrable to base the estmator of the varace fucto o the resduals from a optmal estmator of the mea. Istead t s more desrable to use estmators of the mea wth mmal bas. I addto our results also correct the optmal rate clamed the prevous lterature. Keywords: Mmax estmato, oparametrc regresso, varace estmato. AMS 000 Subject Classfcato: Prmary: 6G08, 6G0. Departmet of Statstcs, The Wharto School, Uversty of Pesylvaa, Phladelpha, PA 904. The research of Toy Ca was supported part by NSF Grat DMS Departmet of Statstcs, Purdue Uversty, West Lafayette, IN

2 Itroducto Cosder the heteroscedastc oparametrc regresso model y fx + V x z,,...,, where x / ad z are depedet wth zero mea, ut varace ad uformly bouded fourth momets. Both the mea fucto f ad varace fucto V are defed o [0, ] ad are ukow. The ma object of terest s the varace fucto V. The estmato accuracy s measured both globally by the mea tegrated squared error R ˆV, V E ad locally by the mea squared error at a pot 0 ˆV x V x dx R ˆV x, V x E ˆV x V x. 3 We wsh to study the effect of the ukow mea f o the estmato of the varace fucto V. I partcular, we are terested the case where the dffculty estmato of V s drve by the degree of smoothess of the mea f. The effect of ot kowg the mea f o the estmato of V has bee studed before Hall ad Carroll 989. The ma cocluso of ther paper s that t s possble to characterze explctly how the smoothess of the ukow mea fucto flueces the rate of covergece of the varace estmator. I assocato wth ths they clam a explct mmax rate of covergece for the varace estmator uder potwse rsk. For example, they state that the classcal rates of covergece 4/5 for the twce dfferetable varace fucto estmator s achevable f ad oly f f s the Lpschtz class of order at least /3. More precsely, Hall ad Carroll 989 stated that, uder the potwse mea squared error loss the mmax rate of covergece for estmatg V s max{ 4α α+, β β+ } 4 f f has α dervatves ad V has β dervatves. We shall show here that ths result s fact correct. I the preset paper we revst the problem the same settg as Hall ad Carroll 989. We show that the mmax rate of covergece uder both the potwse squared error ad global tegrated mea squared error s max{ 4α, β β+ } 5

3 f f has α dervatves ad V has β dervatves. The dervato of the mmax lower boud s volved ad s based o a momet matchg techque ad a two-pot testg argumet. A key step s to study a hypothess testg problem where the alteratve hypothess s a Gaussa locato mxture wth a specal momet matchg property. The mmax upper boud s obtaed usg kerel smoothg of the squared frst order dffereces. Our results have two terestg mplcatos. Frstly, f V s kow to belog to a regular parametrc model, such as the set of postve polyomals of a gve order, the cutoff for the smoothess of f o the estmato of V s /4, ot / as stated Hall ad Carroll 989. That s, f f has at least /4 dervatve the the mmax rate of covergece for estmatg V s solely determed by the smoothess of V as f f were kow. O the other had, f f has less tha /4 dervatve the the mmax rate depeds o the relatve smoothess of both f ad V ad wll be completely drve by the roughess of f. Secodly, cotrary to the commo practce, our results dcate that t s ofte ot desrable to base the estmator ˆV of the varace fucto V o the resduals from a optmal estmator ˆf of f. I fact, the result shows that t s desrable to use a ˆf wth mmal bas. The ma reaso s that the bas ad varace of ˆf have qute dfferet effects o the estmato of V. The bas of ˆf caot be removed or eve reduced the secod stage smoothg of the squared resduals, whle the varace of ˆf ca be corporated easly. The paper s orgazed as follows. Secto presets a upper boud for the mmax rsk whle Secto 3 derves a rate-sharp lower boud for the mmax rsk uder both the global ad local losses. The lower ad upper bouds together yeld the mmax rate of covergece. Secto 4 dscusses the obtaed results ad ther mplcatos for practcal varace estmato the oparametrc regresso. The proofs are gve Secto 6. Upper boud I ths secto we shall costruct a kerel estmator based o the square of the frst order dffereces. Such ad more geeral dfferece based kerel estmators of the varace fucto have bee cosdered, for example, Müller ad Stadtmüller 987 ad 993. For estmatg a costat varace, dfferece based estmators have a log hstory. See vo Neuma 94, 94, Rce 984, Hall, Kay ad Tttergto 990 ad Muk, Bssatz, Wager ad Fretag

4 Defe the Lpschtz class Λ α M the usual way: Λ α M {g : for all 0 x, y, k 0,..., α, g k x M, ad g α x g α y M x y α } where α s the largest teger less tha α ad α α α. We shall assume that f Λ α M f ad V Λ β M V. We say that the fucto f has α dervatves f f Λ α M f ad V has β dervatves f V Λ β M V. For,,...,, set D y y +. The oe ca wrte D fx fx + + V x z V x+ z + δ + V ɛ 6 where δ fx fx +, V V x + V x + ad has zero mea ad ut varace. ɛ V x + V x + V x z V x+ z + We costruct a estmator ˆV by applyg kerel smoothg to the squared dffereces D whch have meas δ + V. Let Kx be a kerel fucto satsfyg Kx s supported o [, ], Kxx dx 0 for,,, β K xdx k <. Kxdx It s well kow kerel regresso that specal care s eeded order to avod sgfcat, sometmes domat, boudary effects. We shall use the boudary kerels wth asymmetrc support, gve Gasser ad Müller 979 ad 984, to cotrol the boudary effects. For ay t [0, ], There exsts a boudary kerel fucto K t x wth support [, t] satsfyg the same codtos as Kx,.e. t t t K t xdx, ad K t xx dx 0 for,,, β K t xdx k < for all t [0, ]. 4

5 We ca also make K t x Kx as t but ths s ot ecessary here. See Gasser, Müller ad Mammtzsch 985. For ay 0 < h <, x [0, ], ad,,, let K h x x +x + / x +x / h x +x + / x +x / x +x + / x +x / x u K h du whe x h, h h K t x u h du whe x th for some t [0, ] h K t x u h du whe x th for some t [0, ] ad we take the tegral from 0 to x + x / for, ad from x + x / to for. The we ca see that for ay 0 x, Kh x. Defe the estmator V as V x K h xd. 7 Same as the mea fucto estmato problem, the optmal badwdth h ca be easly see to be h O /+β for V Λ β M V. For ths optmal choce of the badwdth, we have the followg theorem. Theorem Uder the regresso model where x / ad z are depedet wth zero mea, ut varace ad uformly bouded fourth momets, let the estmator V be gve as 7 wth the badwdth h O /+β. The there exsts some costat C 0 > 0 depedg oly o α, β, M f ad M V such that for suffcetly large, sup sup f Λ α M f,v Λ β M V 0x E V x V x C 0 max{ 4α, β +β } 8 ad sup E f Λ α M f,v Λ β M V 0 V x V x dx C 0 max{ 4α, β +β }. 9 Remark : The uform rate of covergece gve 8 yelds mmedately the potwse rate of covergece that for ay fxed pot x [0, ] sup E V x V x C 0 max{ 4α, β +β }. f Λ α M f,v Λ β M V Remark : It s also possble to use the local lear regresso estmator stead of the Prestley-Chao kerel estmator. I ths case, the boudary adjustmet s ot ecessary as t s well kow that the local lear regresso adjusts automatcally boudary regos, preservg the asymptotc order of the bas tact. However, the proof s slghtly more techcally volved whe usg the local lear regresso estmator. For detals, see, for example, Fa ad Gjbels

6 Remark 3: It s mportat to ote here that the results gve Theorem ca be easly geeralzed to the case of radom desg. I partcular, f the observatos X,..., X are d wth the desg desty fx that s bouded away from zero.e. fx δ > 0 for all x [0, ], the the results of Theorem are stll vald codtoally. I other words, sup sup f Λ α M f,v Λ β M V 0x E V x V x X,..., X C 0 max{ 4α, β +β } + o p max{ 4α, β +β } ad sup E V x V x dx X,..., X f Λ α M f,v Λ β M V 0 C 0 max{ 4α, β +β } + o p max{ 4α, β +β } where the costat C 0 > 0 ow also depeds o δ. 3 Lower Boud I ths secto, we derve a lower boud for the mmax rsk of estmatg the varace fucto V uder the regresso model. The lower boud shows that the upper boud gve the prevous secto s rate-sharp. As Hall ad Carroll 989 we shall assume the lower boud argumet that the errors are ormally dstrbuted,.e., z d N0,. Theorem Uder the regresso model wth z d N0,, f V ad for ay fxed x 0, f V sup E V V C max{ 4α, β +β } 0 f Λ α M f,v Λ β M V sup E V x V x C max{ 4α, β +β } f Λ α M f,v Λ β M V where C > 0 s a costat depedg oly o α, β, M f ad M V. It follows mmedately from Theorems ad that the mmax rate of covergece for estmatg V uder both the global ad local losses s max{ 4α, β +β }. 6

7 show The proof of ths theorem ca be aturally dvded to two parts. The frst step s to f V sup E V x V x C β +β. f Λ α M f,v Λ β M V Ths part s stadard ad relatvely easy. Brow ad Leve 006 cotas a detaled proof of ths asserto for the case β. Ther argumet ca be easly geeralzed to other values of β. We omt the detals. The proof of the secod step, f V sup E V x V x C 4α, 3 f Λ α M f,v Λ β M V s much more volved. The dervato of the lower boud 3 s based o a momet matchg techque ad a two-pot testg argumet. Oe of the ma steps s to study a complcated hypothess testg problem where the alteratve hypothess s a Gaussa locato mxture wth a specal momet matchg property. More specfcally, let X,..., X d P ad cosder the followg hypothess testg problem betwee H 0 : P P 0 N0, + θ ad H : P P Nθ ν, Gdν where θ > 0 s a costat ad G s a dstrbuto of the mea ν wth compact support. The dstrbuto G s chose such a way that, for some postve teger q depedg o α, the frst q momets of G match exactly wth the correspodg momets of the stadard ormal dstrbuto. The exstece of such a dstrbuto s gve the followg lemma from Karl ad Studde 966. Lemma For ay fxed postve teger q, there exst a B < ad a symmetrc dstrbuto G o [ B, B] such that G ad the stadard ormal dstrbuto have the same frst q momets,.e. B B x j Gdx + x j ϕxdx, j,,, q where ϕ deotes the desty of the stadard ormal dstrbuto. The momet matchg property makes the testg betwee the two hypotheses dffcult. The lower boud 3 the follows from a two-pot argumet wth a approprately chose θ. Techcal detals of the proof are gve Secto 6. 7

8 Remark 4: For α betwee /4 ad /8, a much smpler proof ca be gve wth a twopot mxture for P whch matches the mea ad varace, but ot the hgher momets, of P 0 ad P. However, ths smpler proof fals for smaller α. It appears to be ecessary geeral to match hgher momets of P 0 ad P. Remark 5: Hall ad Carroll 989 gave the lower boud C max{ 4α/+α, β/+β } for the mmax rsk. Ths boud s larger tha the lower boud gve our Theorem ad s correct. Ths s due to a mscalculato o appedx C of ther paper. A key step that proof s to fd some d 0 such that { D E [ + exp d + d N ] } d + d N 0. I the above expresso, N deotes a stadard ormal radom varable. But fact D d + d x + exp d + d x exp x π dx x exp x d dx + expx πd d x exp x expx/ + exp x/ πd d d 8 dx. Ths s a tegral of a odd fucto whch s detcally 0 for all d. 4 Dscusso Varace fucto estmato regresso s more typcally based o the resduals from a prelmary estmator ˆf of the mea fucto. Such estmators have the form ˆV x w xy ˆfx 4 where w x are weght fuctos. A atural ad commo approach s to subtract 4 a optmal estmator ˆf of the mea fucto fx. See, for example, Hall ad Carroll 989, Neuma 994, Ruppert, Wad, Holst, ad Hössjer 997, ad Fa ad Yao 998. Whe the ukow mea fucto s smooth, ths approach ofte works well sce the bas ˆf s eglgble ad V ca be estmated as well as whe f s detcally zero. However, whe the mea fucto s ot smooth, usg the resduals from a optmally smoothed ˆf wll lead to a sub-optmal estmator of V. For example, Hall ad Carroll 989 used a kerel estmator wth optmal badwdth for ˆf ad showed that the resultg 8

9 varace estmator attas the rate of max{ 4α α+, β β+ } 5 over f Λ α M f ad V Λ β M V. Ths rate s strctly slower tha the mmax rate whe 4α α+ < β β β+ or equvaletly, α < β+. Cosder the example where V belogs to a regular parametrc famly, such as {V x expax + b : a, b R}. As Hall ad Carroll have oted ths case s equvalet to the case of β results lke Theorems ad. The the rate of covergece for ths estmator becomes oparametrc at 4α α+ for α <, whle the optmal rate s the usual parametrc rate for all α 4 ad s 4α for 0 < α < 4. The ma reaso for the poor performace of such a estmator the o-smooth settg s the large bas ˆf. A optmal estmator ˆf of f balaces the squared bas ad varace. o the estmato of V. However, the bas ad varace of ˆf have sgfcatly dfferet effects The bas of ˆf caot be further reduced the secod stage smoothg of the squared resduals, whle the varace of ˆf ca be corporated easly. For f Λ α M f, the maxmum bas of a optmal estmator ˆf s of order α+ whch becomes the domat factor the rsk of ˆV whe α < β β+. To mmze the effect of the mea fucto such a settg oe eeds to use a estmator ˆfx wth mmal bas. Note that our approach s, effect, usg a very crude estmator ˆf of f wth fx y +. Such a estmator has hgh varace ad low bas. As we have see Secto the large varace of ˆf does ot pose a problem terms of rates for estmatg V. Hece for estmatg the varace fucto V a optmal ˆf s the oe wth mmum possble bas, ot the oe wth mmum mea squared error. Here we should of course exclude the obvous, ad ot useful, ubased estmator ˆfx y. Aother mplcato of our results s that the ukow mea fucto does ot have ay frst-order effect for estmatg V as log as f has more tha /4 dervatves. Whe α > /4, the varace estmator V s essetally adaptve over f Λ α M f for all α > /4. I other words, f f s kow to have more tha /4 dervatves, the varace fucto V ca be estmated wth the same degree of frst-order precso as f f s completely kow. However, whe α < /4, the rate of covergece for estmatg V s etrely determed by the degree of smoothess of the mea fucto f. α 9

10 5 Numercal results We ow cosder ths secto the fte sample performace of our dfferece-based method for estmatg the varace fucto. I partcular we are terested comparg the umercal performace of the dfferece-based estmator wth the resdual-based estmator of Fa ad Yao 998. The umercal results show that the performace of the dfferece-based estmator s somewhat feror whe the ukow mea fucto s very smooth. O the other had, the dfferece-based estmator performs sgfcatly better tha the resdual-based estmator whe the mea fucto s ot smooth. Cosder the model where the varace fucto s V x x + whle there are four possble mea fuctos:. f x 0. f x 3 4 s0π x 3. f 3 x 3 4 s0π x 4. f 4 x 3 4 s40π x. The mea fuctos are arraged from a costat to much rougher susod fucto; the roughess the dffculty a partcular mea fucto creates estmato of the varace fucto V s measured by the fuctoal Rf [f x] dx sce the mearelated term the asymptotc bas of the varace estmator ˆV x s drectly proportoal to t. The umercal performace of the dfferece-based method had bee vestgated earler Leve 006 for a slghtly dfferet set of mea fuctos. For comparso purposes, the same four combatos of the mea ad varace fuctos are vestgated usg the two-step method descrbed Fa ad Yao 998. We expect ths method to perform better tha the dfferece-based method the case of a costat mea fucto, but to get progressvely worse as the roughess of the mea fucto cosdered creases. The followg table summarzes results of smulatos usg both methods. I ths case, the badwdths for estmatg the mea ad varace fuctos were selected usg a K-fold cross-valdato wth K 0. We cosder the fxed equdstat desg x o [0, ] where the sample sze s 000; 00 smulatos are performed ad the badwdth h s selected usg a K-fold cross-valdato wth K 0. The performace of both methods s measured usg the cross-valdato dscrete mea 0

11 squared error CDMSE that s defed as wth h CV CDMSE [ ] ˆVhCV x V x 6 beg the K-fold cross-valdato badwdth. We report the meda CDMSE for varace fucto estmators based o 00 smulatos. The followg table provdes the summary of the performace. Table : Performace uder the chagg curvature of the mea fucto Meda CDMSE Mea fucto Rf Fa-Yao method Our method f f 3 4 s0πx f 3 4 s0πx f 3 4 s40πx It s easly see from the above table that the two-step method of Fa ad Yao, based o estmatg the varace usg squared resduals from the mea fucto estmato, teds to perform slghtly better whe the mea fucto s very smooth but otceably worse whe t s rougher. Note that here we oly use the frst-order dffereces. The performace of the dfferece based estmator ca be mproved the case of smooth mea fucto by usg hgher order dffereces. The Fa-Yao method performs about 6% better the frst case of the costat mea fucto. However, the rsk CDMSE of the dfferece based method s over 95% smaller tha the rsk of the Fa-Yao method for the secod mea fucto. I the rougher cases, the dfferece s approxmately the same. The CDMSE of the dfferece based method s over 95% ad 96% less tha the correspodg rsk of the resdual based method for the thrd ad fourth mea fuctos, respectvely. 6 Proofs 6. Upper Boud: Proof of Theorem We shall oly prove 8. Iequalty 9 s a drect cosequece of 8. Recall that D δ + V + V ɛ + δ V ɛ

12 where δ fx fx +, V V x + V x + ad ɛ V x + V x + V x z V x+ z +. Wthout loss of geeralty, suppose h /+β. It s easy to see that for ay x [0, ], Kh x, ad whe x x + x + / + h or x x + x / h, K hx equals 0. Suppose k < k, we also have K h x h K h x K udu k where K u Ku whe x h, h; K u K t u whe x th for some t [0, ]; ad K u K t u whe x th for some t [0, ]. The secod equalty above s obtaed as follows. For the sake of smplcty, assume that K K; the same argumet ca be repeated for boudary kerels as well. Usg the defto of K hx, we ote that t ca be rewrtte as x+x + x +x h K x u h du. Sce the last tegral s take wth respect to the probablty measure o the terval [ x +x, x +x + ], we ca apply Jese s equalty to obta K h x h Thus, x +x + x +x K h x h x u K du h h x +x + x +x x +x + x +x x u K du. h x u K du K udu. h For all f Λ α M f ad V Λ β M V, the mea squared error of ˆV at x satsfes E V x V x E E + 4 { + 4E K h x D V x K h x δ + K h x V ɛ + K h x δ + 4 K h x V V x K h x δ V ɛ } K h x V V x K h x V ɛ + 4E K h x δ V ɛ.

13 Suppose α /4, otherwse 4α < β/+β for ay β. Sce for ay, δ fx fx + M f x x + α M f α, we have 4 K h x δ 4 K h x M f α km f 4 4α. Note that for ay x, y [0, ], Taylor s theorem yelds β V j y V x V y x y j x x u β j! V β u V β ydu j y β! x x u β M V x y β β du β! y M V β! x y β. ad So, V V x + V + V β V j x j! x j + M V x β + M V V V x β V j x j! x j M V x β M V V x j x j x x β x β. j j Sce the kerel fuctos have vashg momets, for j,,, β, whe large 3

14 eough 0 j K h x x x +x + / x +x / h K x u h x +x + / x +x / x c +x + / x +x / h K x u h u x j du + j x du x +x + / x +x / h K x u h [ ] h K x u j h x u x j du x u j h h du c for some geerc costat c > 0. Smlarly Kh x + x j c. So, β K h x V j x j + j j! x + x Ĉ j [ ] j x u x j for some costat Ĉ > 0 whch does ot deped o x. Note that V β satsfes Hölder codto wth expoet 0 < α α α < ad s, therefore, cotuous o [0, ] ad bouded. The we have 4 K h x V V x Ĉ + MV Ĉ + MV x +h + x h x +h + x h K h x x K h x h + Ĉ β M V β/+β k. β β + + x β + h + β The last equalty s due to the fact 0 < h + < h + < 3h. O the other had, otce that ɛ, ɛ 3, ɛ 5, are depedet ad ɛ, ɛ 4, ɛ 6, are depedet, we have 4E K h x δ V ɛ 4V ar K h x δ V ɛ 6 4 x +h + x h K h x δ V 6M f M V α β/+β k du

15 ad 4E K h x V ɛ 4V ar K h x V ɛ 8MV µ 4 K h x 8M V µ 4 h k 8M V µ 4 β/+β k where µ 4 deotes the uform boud for the fourth momets of the ɛ. Puttg the four terms together we have, uformly for all x [0, ], f Λ α M f ad V Λ β M V E V x V x km f 4 4α + Ĉ β MV β/+β k + 8MV µ 4 β/+β k + 6M f M V α β/+β k C 0 max{ 4α, β/+β } for some costat C 0 > 0. Ths proves Lower Boud: : Proof of Theorem We shall oly prove the lower boud for the potwse squared error loss. The same proof wth mor modfcatos mmedately yelds the lower boud uder tegrated squared error. Note that, to prove equalty 3, we oly eed to focus o the case where α < /4, otherwse β/+β s always greater tha 4α for suffcetly large ad the 3 follows drectly from. For a gve 0 < α < /4, there exsts a teger q such that q + α >. coveece we take q to be a odd teger. From lemma, there s a postve costat B < ad a symmetrc dstrbuto G o [ B, B] such that G ad N0, have the same frst q momets. Let r,,...,, be depedet varables wth the dstrbuto G. Set θ M f B α, f 0 0, V 0 x + θ ad V x. Let gx x for x [, ] ad 0 otherwse. Defe the radom fucto f by f x θ r gx x I0 x. The t s easy to see that f s Λ α M f for all realzatos of r. Moreover, f x θ r are depedet ad detcally dstrbuted. 5 For

16 Now cosder testg the followg hypotheses, H 0 : y f 0 x + V 0 x ν,,...,, H : y f x + V x ν,,...,, where ν are depedet N0, varables whch are also depedet of the r s. Deote by P 0 ad P the jot dstrbutos of y s uder H 0 ad H, respectvely. Note that for ay estmator V of V, max{e V x V 0 x, E V x V x } 6 ρ4 P 0, P V 0 x V x 6 ρ4 P 0, P M 4 f 6B 4 4α 7 where ρp 0, P s the Hellger affty betwee P 0 ad P. See, for example, Le Cam 986. Let p 0 ad p be the probablty desty fucto of P 0 ad P wth respect to the Lebesgue measure µ, the ρp 0, P p0 p dµ. The mmax lower boud 3 follows mmedately from the two-pot boud 7 f we show that for ay, the Hellger affty ρp 0, P C for some costat C > 0. C may deped o q, but does ot deped o. Note that uder H 0, y N0, + θ ad ts desty d 0 ca be wrtte as t d 0 t ϕ ϕ t vθ ϕvdv. + θ + θ Uder H, the desty of y s d t ϕt vθ Gdv. It s easy to see that ρp 0, P d 0 d dµ, sce the y s are depedet varables. Note that the Hellger affty s bouded below by the total varato affty, d0 td tdt d 0 t d t dt. Taylor expaso yelds ϕt vθ v k θ k H k t k! k0 where H k t s the correspodg Hermte polyomal. Ad from the costructo of the dstrbuto G, v Gdv v ϕvdv for 0,,, q. 6

17 So, d 0 t d t ϕt vθ Gdv 0 q+ q+ H t v θ! Gdv ϕt vθ ϕvdv 0 H t v θ! Gdv H t v θ! Gdv + H t! q+ q+ v θϕvdv v θϕvdv H t v θ! ϕvdv. 8 H t! Suppose q + p for some teger p, t ca be see that H t v θ! Gdv H t q+! θ v Gdv H t! θ v Gdv H t! θ B ad q+ H t! v θϕvdv H t! θ v ϕvdv H t! θ v ϕvdv H tθ! H t! θ. So from 8, d 0 t d t H t! B + H t! θ θ 7

18 ad the d0 td tdt H t! H t! θ B + H t! θ B dt θ H t! dt θ dt. 9 Sce t φtdt!! where!! 3 3, for the Hermte polyomal H we have [ H t dt!! + k [!! +!!!!!!!! k k k k ] k k + t k dt k! ] k k + t k dt k! k k + k! t k dt k k + k!! k! k + k! For suffcetly large, θ < / ad t the follows from the above equalty that H t! θ B θ B θ B dt H t dt!!!! θ p B θ p! θ p e B ad H t! θ dt θ p θ! H t dt θ!!! θ p θ p p+. θ p θ p!! θ p! p 8

19 The from 9 d0 td tdt θ p eb + p cθ q+ where c s a costat that oly depeds o q. So d0 ρp 0, P td tdt cθ q+ c αq+. Sce αq +, lm c αq+ e c > 0 ad the theorem the follows from 7. Ackowledgmet We thak the Edtor ad three referees for thorough ad useful commets whch have helped to mprove the presetato of the paper. We would lke to thak Wllam Studde for dscussos o the fte momet matchg problem ad for the referece Karl ad Studde 966. Refereces [] Brow, L. D. ad Leve, M Varace estmato oparametrc regresso va the dfferece sequece method. A. Statst., press. [] Fa, J. ad Gjbels, I Local Polyomal Modellg ad Its Applcatos. Chapma ad Hall, Lodo. [3] Fa, J. ad Yao, Q Effcet estmato of codtoal varace fuctos stochastc regresso. Bometrka 85, [4] Hall, P. ad Carroll, R. J Varace fucto estmato regresso: the effect of estmatg the mea. J. Roy. Statst. Soc. Ser. B 5, 3-4. [5] Müller, H. G. ad Stadtmüller, U Estmato of heteroscedastcty regresso aalyss. A. Statst. 5, [6] Müller, H. G. ad Stadtmüller, U O varace fucto estmato wth quadratc forms. J. Stat. Plag ad If. 35, 3-3. [7] Gasser, T. ad Müller, H. G Kerel estmato of regresso fuctos. I Smoothg Techques for Curve Estmato, pp Berl: Sprger. Lecture Notes Mathematcs No

20 [8] Gasser, T. ad Müller, H. G Estmatg regresso fuctos ad ther dervatves by the kerel method. Scad. J. Statst., 97-. [9] Gasser, T., Müller, H. G. ad Mammtzsch, V Kerels for oparametrc curve estmato. J. Roy. Statst. Soc. B 47, [0] Hall, P., Kay, J. ad Tttergto, D. M Asymptotcally optmal dfferecebased estmato of varace oparametrc regresso. Bometrka 77, [] Karl, S. ad Studde, W. J Tchebycheff Systems: Wth Applcatos I Aalyss Ad Statstcs. Iterscece, New York. [] Le Cam, L Asymptotc Methods Statstcal Decso Theory. Sprger- Verlag, New York. [3] Muk, A., Bssatz, N., Wager, T. ad Fretag, G O dfferece based varace estmato oparametrc regresso whe the covarate s hgh dmesoal. J. Roy. Statst. Soc. B 67, 9-4. [4] Leve, M Badwdth selecto for a class of dfferece-based varace estmators the oparametrc regresso: A possble approach. Comput. Statst. Data Aal. 50, [5] Neuma, M Fully data-drve oparametrc varace estmators. Statstcs 5, 89-. [6] Parze, E. 96. O estmato of a probablty desty fucto ad mode. A. Math. Statst. 33, [7] Rce, J Badwdth choce for oparametrc kerel regresso. A. Statst., [8] Roseblatt, M Remarks o some oparametrc estmates of a desty fucto. A. Math. Statst. 7, [9] Ruppert, D., Wad, M. P., Holst, U. ad Hössjer, O Local polyomal varace fucto estmato. Techometrcs 39, [0] Stoe, C. J Optmal rates of covergece for oparametrc estmators. A. Statst. 8,

21 [] vo Neuma, J. 94. Dstrbuto of the rato of the mea squared successve dfferece to the varace. A. Math. Statst., [] vo Neuma, J. 94. A further remark cocerg the dstrbuto of the rato of the mea squared successve dfferece to the varace. A. Math. Statst. 3,