Adaptive Estimation of Density Function Derivative

Transcription

1 Applied Methods of Statistical Aalysis. Noparametric Approach Adaptive Estimatio of Desity Fuctio Derivative Dimitris N. Politis, Vyacheslav A. Vasiliev 2 ad Peter F. Tarasseko 2 Departmet of Mathematics, Uiversity of Califoria, Sa Diego, USA 2 Departmet of Applied Mathematics ad Cyberetics, Tomsk State Uiversity, Tomsk, Russia dpolitis@ucsd.edu, vas@mail.tsu.ru, ptara@mail.tsu.ru Abstract The properties of o-parametric kerel estimators for the first order derivative of probability desity fuctio from special parameterized classes are ivestigated. I particular, i the case of kow smooth classes parameter, rates of mea square covergecy of desity ad its derivative estimators of smooth parameter estimators are foud. Adaptive estimators of desities ad their first derivatives from the give class with the ukow smooth parameter are costructed. No-asymptotic ad asymptotic properties of these estimators are established. Keywords: No-parametric kerel desity estimators, smooth parameter estimatio; adaptive desity derivative estimators, mea square covergece, rate of covergece, smoothess class. Itroductio Let X,..., X be idepedet idetically distributed radom variables (i.i.d. r.v. s) havig a probability desity fuctio f. I the typical oparametric set-up, othig is assumed about f except that it possesses a certai degree of smoothess, e.g., that it has r cotiuous derivatives. Estimatig f via kerel smoothig is a sixty year old problem; M. Roseblatt who was oe of its origiators discusses the subject s history ad evolutio i the moograph by [3]. For some poit x, the kerel smoothed estimator of f(x) is defied by f,h (x) = x j= h K Xj, () h where the kerel K is a bouded fuctio satisfyig K(x)dx = ad K 2 (x)dx <, ad the positive badwidth parameter h is a decreasig fuctio of the sample size. I this paper we will employ a particularly useful class of ifiite order kerels amely the flat-top family; see [7] for a geeral defiitio. It is a well-kow fact that optimal badwidth selectio is perhaps the most crucial issue i such oparametric smoothig problems; see [3] ad the refereces therei. The goal typically is miimizatio of the large-sample Mea Squared Error (MSE) of f,h (x). However, to do this miimizatio, the practitioer eeds to kow the degree of smoothess r. Usig a ifiite order kerel ad focusig just o optimizig the order of magitude of the large-sample MSE, it is

2 Sessio apparet that the optimal badwidth h must be asymptotically of order /(2r+) that yields a large-sample MSE of order 2r/(2r+) (see, e.g., [2]). The problem of course is that, as previously metioed, the uderlyig degree of smoothess r is typically ukow. I Sectio 3 of the paper at had, we develop a estimator r of r ad prove its strog cosistecy. I order to costruct our estimator r, we operate uder a class of fuctios that is slightly more geeral tha, e.g., the Hölder class; this class of fuctios is formally defied i Sectio via eq. (3) or (4). Uder such a coditio o the tails of the characteristic fuctio we are able to show i Sectio 2 that the optimized MSE of ˆf (x) is agai of order 2r/(2r+) for possibly oiteger r. Furthermore, i Sectio 4 we develop a adaptive estimator ˆf (x) that achieves the optimal MSE rate of 2r/(2r+) withi a logarithmic factor despite the fact that r is ukow, see Examples after Theorem 3. Similar effect arises i the adaptive estimatio problem of the desities, i particular, from the Hölder class, see [, 4, 5]. The estimato problem of the desity derivatives is actual as well; i particular for estimatio of the logarithmic derivative. As the major theoretical result of our paper, we are able to prove a o-asymptotic upper boud for the MSE of the adaptive estimator of the desity f ad f. The rate of covergecy i the mea square sese satisfies (for the estimators of f i examples) the abovemetioed optimal rate. Sectio 5 cotais some simulatio results showig the performace of the estimator ˆf (x) i practice. Full ivestigatio of the desity fuctio estimators will be preseted i the paper [2]. Problem set-up ad basic assumptios Let X,..., X be i.i.d. havig a probability desity fuctio f. Deote ϕ(s) = e isx f(x)dx the characteristic fuctio of f ad the sample characteristic fuctio ϕ (s) = e isx k. For some fiite r > 0, defie two families F + r ad F r of bouded, i.e., 0 < f < : sup y R f(y) f, (2) ad cotiuous fuctios f satisfyig oe of the followig coditios respectively: s r ϕ(s) ds <, s r+ε ϕ(s) ds =, for all ε > 0, (3) k= s r ε ϕ(s) ds <, s r ϕ(s) ds =, for all 0 < ε < r. (4) It is easy to verify that the derivative f satisfies the relatios (3) ad (4) if f F + r+ ad f F r+ respectively. 2

3 Applied Methods of Statistical Aalysis. Noparametric Approach Defie the family F r,m + (respectively F r,m ) as the family of fuctios f from F r + (respectively F r ) but with f beig such that its characteristic fuctio ϕ(s) has mootoously decreasig tails. Cosider the class Ξ of kerel smoothed estimators f,h (x) of f(x) as give i eq. (). Note that we ca alteratively express f (l),h (x) i terms of the Fourier trasform of kerel K, i.e., f (l),h (x) = x Xj K(l) = λ (l) (s, h)ϕ h+l (s)e isx ds, l = 0;, (5) h 2π j= where λ (0) (s, h) = K x h e isx dx ad λ () (s, h) = K ( x h) e isx dx = ishλ (0) (s, h). I this paper, we will employ the family of flat-top ifiite order kerels, i.e., we will let the fuctio λ (0) (s, h) be of the form if s /h, λ c (s, h) = g(s, h) if /h < s c/h, 0 if s c/h, where c is a fixed umber i [, ) chose by the practitioer, ad g(s, h) is some properly chose cotiuous, real-valued fuctio satisfyig g(s, h) = g( s, h), g(s, ) = g(s/h, h), ad g(s, h), for ay s, with g(/h, h) =, ad g(c/h, h) = 0; see [7]-[0] for more details o the above flat-top family of kerels. Deote for every 0 γ < r the fuctios δ γ (h) = s r γ ϕ(s) ds, whe h > 0, ad δ γ (0) = 0. /h< s <c/h From (3) ad (5) it follows that δ γ (h) = o() as h 0 for f F r + ad γ = 0, as well as for f F r ad 0 < γ < r. I other cases δ γ (h) =. Defie the followig classes F r = F r + F r ad F r,m = F r,m + F r,m. The mai aim of the paper is adaptive estimatio of desities ad their first derivatives from the class F r with the ukow r. 2 Asymptotic mea square optimal estimatio of f Accordig to [0, ] the mea square error (MSE) u 2 f (f,h) = E f (f,h (x) f(x)) 2 of the estimators f,h (x) Ξ, f F r has the followig form: u 2 f(f,h ) = Uf 2 (h, c) ( 2 K(v)f(x hv)dv), (6) where Uf 2 (h, c) is the pricipal term of the MSE, Uf 2 (h, c) = L f(x) + h 2π /h< s <c/h ( g(s, h))ϕ(s)e isx ds 2, 3

4 Sessio L = K 2 (v)dv. Thus, i particular, sup f F r K(v)f(x hv)dv <. The optimal (i the mea square sese) value h 0 = h 0 () is defied from miimizatio of the pricipal term Uf 2 (h, c). Defie the umber h 0 = h 0 () from the equality (h 0 ) 2r+ 2γ δ 2 γ(h 0 ) = π2 L f(x) (c 0 + c (γ)). (7) I such a way we have proved the followig theorem, which gives the rates of covergece of the radom quatities f 0 (x) = f,h 0(x) ad f,h 0 (x). We ca loosely call f 0 (x) ad f,h 0 (x) estimators although it is clear that these fuctios ca ot be cosidered as estimators i the usual sese i view of the depedece of the badwidths h 0 ad h 0 o ukow parameters r ad f(x). Nevertheless, this theorem ca be used for the costructio of boa fide adaptive estimators with the optimal ad suboptimal coverges rates; see, e.g., Examples ad 2 i what follows. Theorem. Let f(x) > 0. The for the asymptotically optimal (with respect to badwidth h) i the MSE sese estimator f(x) 0 of the fuctio f F r ad for the estimator f,h 0 (x) of f F r,m the followig limit relatios, as, hold. sup if h u2 f (f,h) Uf 2(h0, c) = O ( ) ; f F r 2. for every f F r,m with γ = 0 if f F r,m + ad every 0 < γ < r if f F r,m, as well as some costat C γ, we have 2 2r+ 2γ u 2 f(f) 0 u 2 f(f,h 0 ) C γ δ γ (h 0 ),. 2r 2γ 2r+ 2γ Theorem ad Theorem 2 below are prove i [2]. Theorem 3 is ew. Remark. The defiitio (7) of h 0 is essetially simpler tha the defiitio of the optimal badwidth h 0. From Theorem it follows, that the (slightly) suboptimal estimator f,h 0 ca be successfully used istead. Example. Cosider a estimatio problem of the fuctio f F r,m, + satisfyig the followig additioal coditio ϕ(s) s r+ l +φ as s, φ > 0. (8) s We fid the rates of covergece of the MSE u 2 f (f 0 ) ad u 2 f (f,h 0 ) : h 0 ( ) l 2(+φ) 2r+ ad usig the piecewise liear flat-top kerel λ LIN c λ LIN c (s, h) = c c ( ad u 2 f(f,h 0 ) = O 2r l 2(+φ) ) 2r+ (s, h), itroduced by [9] (see [0] as well) ( h c s ) + c ( h s )+, 4

5 Applied Methods of Statistical Aalysis. Noparametric Approach where (x) + = max(x, 0) is the positive part fuctio, we fid h 0 l 2(+φ) 2(r+) ( ) ad u 2 f(f) 0 2(r+) = O = o u 2 f(f 2r+ l 2(+φ),h 0 ). Example 2. Cosider a estimatio problem of the fuctio f F r,m, satisfyig the followig additioal coditio: ϕ(s) s r+ as s. We fid the rate of covergece of the MSE u 2 f (f ) 0 ad u 2 f (f,h 0 ). From (7) we have ) h 0 2r+ ad u 2 f(f,h 0 ) = O ( 2r 2r+, as. Similarly to Example, as, for f F r we fid h 0 2(r+) ad u 2 f(f 0 ) = O ( 2r+ 2(r+) Similar results ca be obtaied for the estimators of f. ) = o u 2 f(f,h 0 ). 3 Estimatio of the degree of smoothess r Defie the fuctios Φ α (A, B) = s α ϕ(s) ds, Φ,α (A, B) = s α ϕ (s) ds. A< s <B A< s <B Let (δ ) ad (ρ ) be two give sequeces of positive umbers chose by the practitioer such that δ 0 ad ρ as. The sequece (δ ) represets the grid -size i our search of the correct expoet r, while (ρ ) represets a upper boud that limits this search. Defie the followig sets of o-radom sequeces C + = {(A, B, δ ) : A, 0 < A < B, δ 0 as ; for some m 0 2, B 2m 0(ϱ ++δ ) m 0 < ; Φ r+ε (A, B ), ε > 0; Φ r+δ (A, B ) }, C = {(A, B, δ ) : A, 0 < A < B, δ 0 as ; for some m 0 2, B 2m 0(ϱ ++δ ) < ; Φ m r δ (A, B ) 0; Φ r (A, B ) } 0 ad for a arbitrary give H > 0 chose by the practitioer, the estimators (r + ) ad (r ) of the parameter r i (3) ad (4) respectively r + = mi[ϱ, (δ if{k : Φ,(k+)δ (A, B ) H, (A, B, δ ) C + })], (9) r = mi[ϱ, (δ if{k : Φ,kδ (A, B ) H, (A, B, δ ) C})]. (0) 5

6 Sessio Theorem 2. The estimators r + ad r, defied i (9) ad (0) respectively have the followig properties a) if f F r + ad for some δ 0 the sequeces (A, B, δ ) C +, the lim δ (r + r) = 0 P f a.s. b) if f F r ad for some δ 0 the sequeces (A, B, δ ) C, the lim δ (r r) = 0 P f a.s. 4 Adaptive estimatio of the fuctios f, f F r The purpose of this sectio is the costructio ad ivestigatio of a adaptive estimator of the fuctios f, f F r with ukow r, which ca either serve as the mai estimator or ca serve as a pilot estimator for the costructio of a adaptive optimal ad suboptimal badwidths ĥ0 ad ĥ0. We defie a adaptive estimators of f ad f from F r as follows ˆf (l) (x) = j= Λ (l) j (x X j) = 2π j= λ (l) j (s)e is(x X j) ds, () where Λ (l) j (z) = K (l) z = λ (l) ĥ +l j j ĥ j 2π (s)e isz ds is the smoothig kerel, ad λ (0) j (s) = λ c (s, ĥj ), l = 0;. The required badwidths are defied by ĥ j = (j + ) +2(r(j)+l), j, where r(j) = r + j if f F + r ad r(j) = r j if f F r ; recall that the estimators r + j ad r j are defied i (9) ad (0) respectively. Below C(γ, l) are some costats ad Ψ γ,l () are cocrete decreasig to zero fuctios. Mai properties of costructed estimators are stated i the followig theorem. Theorem 3. Let the sequeces (A, B, δ ) i the defiitio of the estimator r + belog to the set C + ad i the defiitio of the estimator r to the set C. Let γ = 0 if f F r + ad γ (0, r) if f F r, as well as r > 0 if l = 0 ad r > if l =. The for every the estimators () has the followig properties: u 2 (l) C(γ, l) f( ˆf ) Ψ γ,l ()+, l = 0;. sup f F r Examples ad 2 revisited. Uder appropriate chose δ > 0 ad sequeces (A, B, δ ) i the defiitio of sets C +, C : 6

7 Applied Methods of Statistical Aalysis. Noparametric Approach Figure : MSE of kerel estimators multiplied by 3/4 versus {25, 2000}. Left chart correspods to the estimator with piece-wise liear kerel characteristic fuctio. Right chart correspods to the estimator with ifiitely-differetiable flat-top kerel characteristic fuctio. I Example (case (f F + r )) Ψ 0,0 () (h ) (l ) 2δ (+2r) 2 2r +2r (l ) 2δ (+2r) 2. The, accordig to Theorem 3, i this case the rate of covergece of adaptive desity estimators of f F r + differs from the rate of o-adaptive estimators i [0] o the extra log-factor oly. For the fuctios f F r ad γ (0, mi(r, )) from Example 2 it follows that Ψ γ () 2(r γ) δ +2(r γ) (l ) +2(r γ) as. 5 Simulatio results I this sectio we provide brief results of simulatio study of the estimators itroduced i Sectio 2. We examie kerel estimators of triagular probability desity fuctio f(x) = (a x )/a 2, x a belogig to the family F with characteristic fuctio ϕ(s) = 2( cos(as))/(as) 2. Also ϕ(s) meets requiremets of the Example 2. Thus the badwidth ca be take i the form h = O( /4 ) ad expected covergece rate of the kerel estimator MSE is 3/4. Two flat-top kerels have bee used i the simulatio. First oe has the piece-wise liear kerel characteristic fuctio itroduced i [0]: λ(s) = {, s ; (c s )/(c ), < s < c; 0, s c}. Secod case refers to the ifiitely-differetiable flat-top kerel characteristic fuctio (see [6]) λ(s) = {, s c; exp [ b exp [ b/( s c) 2 ] /( s ) 2 ], c < s < ; 0, s }. The mai goal of the simulatio study is ivestigatio of the MSE behavior for the kerel estimator with the growth of sample size. We geerate sequece of 50 samples for each sample size from 25 to 2000 with step 25, the calculate the estimator MSE multiplied by 3/4 ad expect visual stabilizatio of the sequece of resultig values with growth of. Two typical examples are preseted at the Figure. Both cases refer to estimatio of triagle desity fuctio f(x) with uit variatio (which support is bouded by ±2.45, a = 2.45) 7

8 Sessio at the poit x =.0 by kerel estimators with flat-top kerels. The expected stabilizatio is observig i both cases. Refereces [] Brow L.D., Low M.G. (992) Superefficiecy ad lack of adaptibility i fuctioal estimatio. Techical report, Corell Uiv. [2] Dobrovidov, A.V., Koshki, G.M., Vasiliev, V.A. (202) No-parametric state space models. Heber City, Utah: Kedrick Press. [3] Joes M.C., Marro J.S., Sheather S.J. (996) A brief survey of badwidth selectio for desity estimatio, J. Amer. Statist. Assoc., vol. 9, [4] Lepski O.V. (990) Oe problem of adaptive estimatio i Gaussia white oise. Theor. Probab. Appl., 35, pp (i Russia). [5] Lepski O.V., Spokoiy V.G. (997) Optimal poitwise adaptive methods i oparametric estimatio. A. Statist., 25 (6), pp [6] McMurry T., Politis D.N. (2004) Noparametric regressio with ifiite order flat-top kerels, J. Noparam. Statist., vol. 6, o. 3-4, , [7] Politis D.N. (200) O oparametric fuctio estimatio with ifiite-order flat-top kerels, i Probability ad Statistical Models with applicatios, Ch. Charalambides et al. (Eds.), Chapma ad Hall/CRC: Boca Rato, pp [8] Politis D.N. (2003) Adaptive badwidth choice, J. Noparam. Statist., vol. 5, o. 4-5, , [9] Politis D.N., Romao J.P. (993) O a Family of Smoothig Kerels of Ifiite Order, i Computig Sciece ad Statistics, Proceedigs of the 25th Symposium o the Iterface, Sa Diego, Califoria, April 4-7, 993, (M. Tarter ad M. Lock, eds.), The Iterface Foudatio of North America, pp [0] Politis D.N., Romao J.P. (999) Multivariate desity estimatio with geeral flat-top kerels of ifiite order. Joural of Multivariate Aalysis, 999, 68, -25. [] Politis D.N., Romao J.P., Wolf M. (999) Subsamplig. Spriger, New York, 999. [2] Politis D.N., Vasiliev V.A., Tarasseko P.F. (205) Estimatig smoothess ad optimal badwidth for probability desity fuctios. Workig paper (submitted). [3] Roseblatt M. (99) Stochastic curve estimatio. NSF-CBMS Regioal Coferece Series, 99, 3, Istitute of Mathematical Statistics, Hayward. 8