Chapter 4 Smoothed Nonparametric Derivative Estimation Based on Weighted Difference Sequences

Transcription

1 Chapter 4 Smoothed Nonparametrc Dervatve Estmaton Based on Weghted Dfference Sequences Krs De Brabanter and Yu Lu Abstract We present a smple but effectve fully automated framework for estmatng dervatves nonparametrcally based on weghted dfference sequences. Although regresson estmaton s often studed more, dervatve estmaton s of equal mportance. For example n the study of exploraton of structures n curves, comparson of regresson curves, analyss of human growth data, etc. Va the ntroduced weghted dfference sequence, we approxmate the true dervatve and create a new data set whch can be smoothed by any nonparametrc regresson estmator. However, the new data sets created by ths technque are no longer ndependent and dentcally dstrbuted (..d.) random varables. Due to the non-..d. nature of the data, model selecton methods tend to produce bandwdths (or smoothng parameters) whch are too small. In ths paper, we propose a method based on bmodal kernels to cope wth the non-..d. data n the local polynomal regresson framework. 4.1 Introducton The popularty of nonparametrc methods have ncreased snce ther ntroducton n the md 1950s and early 1960s. One of the man reasons for ther popularty s the flexblty these methods possess. Snce ther ntroducton, many of ther propertes have been rgorously nvestgated, see e.g. [6]. Most of the propertes have been establshed for nonparametrc regresson estmaton, but not as much for nonparametrc dervatve estmaton even though the dervatve of the regresson estmate s of great mportance as well (e.g. nference about slopes of the regresson estmates). See e.g. comparson of regresson curves [8], trend analyss n tme seres [11], the exploraton of structures n curves [1], analyss of human growth data [10], etc. K. De Brabanter (B) Department of Statstcs and Department of Computer Scence, Iowa State Unversty, 2419 Snedecor Hall, Ames, IA, , USA e-mal: kbrabant@astate.edu Y. Lu Department of Computer Scence, Iowa State Unversty, Atanasoff Hall, Ames, IA, USA e-mal: yulu@astate.edu Sprnger Internatonal Publshng Swtzerland 2015 A. Steland et al. (eds.), Stochastc Models, Statstcs and Ther Applcatons, Sprnger Proceedngs n Mathematcs & Statstcs 122, DOI / _4 31

2 32 K. De Brabanter and Y. Lu In general there exst two approaches to nonparametrc dervatve estmaton: Regresson/smoothng splnes and local polynomal regresson. Splne dervatve estmators can acheve the optmal L 2 rate of convergence [13]. Further asymptotc theoretcal propertes (bas, varance and normalty) were studed by [15]. However, to ntroduce more flexblty n the smoothng process and to overcome the choosng of the knots, smoothng splnes are a very attractve method for dervatve estmaton. However, choosng the smoothng parameter s stll dffcult [9]. Accordng to [9], data-drven methods are n general not the rght way to deal wth these problems and user nterventon s recommended. Also, the smoothng parameter for a smoothng splne depends on the nteger q whle mnmzng n =1 ( ˆm (q) (x ) m (q) (x )) 2 [14]. In the context of kernel regresson estmaton, [7] proposed a generalzed verson of the cross-valdaton technque to estmate the frst dervatve va kernel smoothng usng dfference quotents. Unfortunately, the varance of dfference quotents s proportonal to the square of sample sze. Consequently, the estmaton wll be rendered useless due to ths large varance. On the other hand, the local polynomal framework [5] offers a nce way of estmatng dervatves. Consder the bvarate data (X 1,Y 1 ),...,(X n,y n ) whch form an ndependent and dentcally dstrbuted (..d.) sample from a populaton (X, Y ). Denote by m(x) = E[Y X] the regresson functon. The data s regarded to be generated from the model Y = m(x) + e, (4.1) where E[e X]=0, Var[e X]=σe 2 < and X and e are ndependent. The am of ths paper s to estmate the dervatve m of the regresson functon m. In ths paper we choose the local polynomal regresson estmator to smooth the data. Ths paper s organzed as follows: Sect. 4.2 gves a short overvew of dervatve estmaton n the fxed desgn settng and extends these results to the random desgn. Secton 4.3 descrbes how to obtan a bandwdth for the local polynomal regresson estmator n case of correlated errors n random desgn. Secton 4.4 provdes a smulaton study of the proposed methodology. Fnally, Sect. 4.5 states the conclusons and dscusses optons for further research. 4.2 Dervatve Estmaton va a Weghted Dfference Sequence Fxed Desgn If X denotes the closed real nterval [a,b] then x = a + ( 1)(b a)/(n 1) and denote d(x ) = b a. In what follows, we assume that the data s ordered.e. x 1 x 2 x n. As mentoned n the ntroducton, for equspaced desgn the use of dfference quotents (Y Y 1 )/(x x 1 ) may be natural, but ther varances are O(n 2 ). Therefore, t s approprate to reduce the varance by usng the followng

3 4 Nonparametrc Dervatve Estmaton 33 weghted symmetrc dfference sequence to obtan the frst order (nosy) dervatve estmator k ( ) Y (1) = Y (1) Y+j Y j (x ) = w,j, (4.2) x +j x j j=1 where the weghts w,1,...,w,k sum up to one. Next, we need to determne the weghts such that the varance s mnmzed. Assume model (4.1) holds wth equspaced desgn and let k j=1 w j = 1. Then, for k + 1 n k, the weghts 6j 2 w,j = w j =, j = 1,...,k (4.3) k(k + 1)(2k + 1) mnmze the varance of Y (1) n (4.2), see [3]. Underthsdesgn settng,assume that m s twce contnuously dfferentable on X R. If the second order dervatve of m s fnte on X, then for the weghts (4.3), t follows that [3] bas ( Y (1) ) ( = O n 1 k ) and Var ( Y (1) ) ( = O n 2 k 3) unformly for k + 1 n k. Explct bas and varance expressons for the nteror and boundary regon are gven n [3]. If k as n such that nk 3/2 0 and n 1 k 0 and under the prevous stated assumptons, we have that for ε>0 P ( (1) Y m (x ) ) ε 0. The tunng factor k can be chosen va the rule of thumb ( 16 ˆσ ˆk 2 ) 1/5 e = (sup x0 X ˆm(2) (x 0 ) ) 2 d(x ) 4 n 4/5, where α s the largest nteger not greater than α. The error varance can be estmated by means of any consstent error varance estmator and sup x0 X ˆm(2) (x 0 ) can be obtaned by fttng a local polynomal regresson estmate of order p = 3 leadng to the followng (rough) estmate of the second dervatve ˆm (2) (x 0 ) = 2 ˆβ 2. By Jensen s nequalty we obtan the L 1 rate of convergence E Y (1) m (x ) = O ( n 1/5). A smlar analyss can be made for hgher order dervatves. We refer the reader to [3] Random Desgn We assume that a densty functon f exsts for the desgn ponts X [a,b] for all = 1,...,n. As before, we assume that the data s ordered.e. X 1 X 2 X n. Smlarly, we need to fnd a sequence of weghts such that the varance of (4.2) s mnmzed. Under the constrant that the weghts have to sum up to 1, we have for k + 1 n k and j = 1,...,k

4 34 K. De Brabanter and Y. Lu Var ( Y (1) ) X+j,...,X j [ ( ) 2 = 2σe 2 1 k k w 2 ],j (X +1 X 1 ) 2 1 w,j + (X j=2 j=2 +j X j ) 2. Settng the partal dervatves to zero and normalzng the weghts such that they sum up to 1, yeld the followng fnte sample weghts w,j = (X +j X j ) 2 kl=1 (X +l X l ). (4.4) 2 For further asymptotc analyss, the term X +j X j needs to rewrtten as a functon of the desgn densty f. Snce our data s sorted, we can use the followng approxmaton X +j X j = 2j nf (X ) + o p(j/n). Assumng m s twce contnuously dfferentable X R, m (2) s fnte on X and usng the weghts (4.4)gvesfork + 1 n k and j = 1,...,k bas ( Y (1) ) X j,...,x +j sup m (2) (x) k x X Var ( Y (1) ) X j,...,x +j = 2σ 2 e j=1 w,j (X +j X j ) 4 = 3k(k + 1) sup x X m(2) (x) 4n(2k + 1)f (X ) ( = O p kn 1 ) k j=1 w 2,j (X +j X j ) 2 + o p ( kn 1 ) = 3σ e 2n2 f 2 (X ) k(k + 1)(2k + 1) + o ( p n 2 k 3) ( = O p n 2 k 3). For a densty f bounded away from zero, m (2) fnte on X, k as n such that nk 3/2 0 and n 1 k 0, t follows that for ε>0, P ( Y (1) m (X ) ε ) 0. The asymptotc mean squared error of the frst order dervatve estmator can be upperbounded by MSE ( Y (1) ) 9k X 2 (k + 1) 2 sup 2 j,...,x +j x X m(2) (x) 16n 2 (2k + 1) 2 f 2 + 3σ e 2n2 f 2 (X ) (X ) k(k + 1)(2k + 1). Mnmzng the above expresson w.r.t. k results n a value for k dependng on X and s gven by ( 16σ 2 k(x ) = e f 4 ) (X ) 1/5 sup 2 n 4/5. (4.5) x X m(2) (x)

5 4 Nonparametrc Dervatve Estmaton 35 Fg. 4.1 Smulated data set of sze n = 250 from model (4.1) wthm(x) = sn 2 {2π(X 0.5)}, X U[0, 1] and e N(0, ).Thefull lne shows the true frst order dervatve, the data ponts show the emprcal frst order dervatves for k {2, 5, 7, 13} In order to obtan an estmator for k(x ), one needs to replace σe 2 and f(x ) by ther consstent estmators. As n fxed desgn, the supremum of the absolute value of the second order dervatve functon can be obtaned by fttng a local cubc ft to the orgnal data. A global k can be obtaned by mnmzng the asymptotc mean ntegrated squared error (AMISE) resultng nto ( 16σ 2 e f 2 ) (x) dx 1/5 k = sup 2 x X m(2) (x) 1 f 2 (x) dx n 4/5. (4.6) The expressons for a varyng and global k can take any postve value. To be used n practce, we round the value down to the smallest nteger. The densty f can be estmated by e.g. kernel densty estmaton. The bandwdth of the kernel for densty estmaton can be obtaned by the solve-the-equaton plug-n bandwdth selector [12]. It mmedately follows that the L 2 rate of convergence n random desgn yelds (for k + 1 n k and j = 1,...,k) E ( Y (1) m (X ) 2 ) ( X j,...,x +j = Op n 1/5 ). It can be shown that usng a local varyng k always leads to an mprovement over a global k. In order to reduce the bas at the boundares, some correctons also need be to be taken nto account. Fgure 4.1 dsplays the emprcal dervatve for k {2, 5, 7, 13} generated from model (4.1) wth m(x) = sn 2 {2π(X 0.5)} where X U[0, 1], n = 250 and e N(0, ).

6 36 K. De Brabanter and Y. Lu 4.3 Smoothng the Nosy Dervatve Data It s clear that for the newly generated data set the ndependence assumpton s no longer vald snce t s a weghted sum of dfferences of the orgnal data set. In such cases, t s known that data-drven bandwdth selectors and plug-ns break down [2]. In [4], the authors extend ths approach to the random desgn settng. In order to ft the newly obtaned data set, we consder the followng model for the frst order dervatve: Y (1) (X) = m (X) + ε where E[ε X]=0and Cov(ε,ε j X,X j ) = σe 2ρ n(x X j ). If the followng holds: f contnuous and bounded away from zero, K(u)du = 1, K 0 and symmetrc, lm u uk(u) =0, sup u K(u) <, ρ n s a statonary, symmetrc correlaton functon wth ρ n (x) 1, x and ρ n (0) = 1. Further assume short range correlaton.e., ξ >0 : ρ n (t) I(h 1 t ξ)dt = o( ρ n (t) dt) and n ρ n (t x) f(t)dt= O(1); then for a kernel K satsfyng K(0) = 0, h 0 and nh as n,the correlaton structure s removed n the model selecton procedure wthout any pror knowledge about ts structure. Snce these bmodal kernels ntroduce extra varance nto the estmate, we develop a relaton between the bandwdth h of a unmodal kernel K and the bandwdth h b of a bmodal kernel K. Consequently, the estmate based on ths bandwdth wll be smoother than the one based on a bmodal kernel. A remarkable consequence of usng a kernel K satsfyng K(0) = 0 s that we do not need to use cross-valdaton (leave-one-out, v-fold, etc.), but smply mnmzng the resdual sum of squares suffces to obtan the value for the bandwdth h! Its easly verfed that for local polynomal regresson (p odd) ĥ = C p (K, K)ĥ b, where [ K 2 p C p (K, K)= (u) du{ u p+1 K ] p (u) du}2 1/(2p+3) K 2 p (u) du{. u p+1 Kp (u) du}2 The factor C p (K, K) s easy to calculate. We take K(u) = (2/ π)u 2 exp( u 2 ) as bmodal kernel. Kp denotes the equvalent kernel, see [5]. For a Gaussan (unmodal) kernel and three dfferent values of p, the factor C p (K, K) equals , and for p = 1, 3, 5 respectvely. 4.4 Smulaton In a frst smulaton, consder the followng two functons m(x) = 1 6X +36X 2 53X X 5 and m(x) = sn(2πx) wth n = 500 generated from a unform dstrbuton on [0, 1]. The error varance was taken to be σ 2 e = 0.05 and e N(0,σ2 e ) for both functons. The value of k was obtaned va (4.6) whch was 6 and 7 respectvely for the frst and second functon (see Fg. 4.2). We used local cubc regresson

7 4 Nonparametrc Dervatve Estmaton 37 Fg. 4.2 Frst order dervatve estmaton. Estmated dervatve by the proposed method (full lne) and true dervatve (dashed lne) for both functons. The value of k (global) was obtaned va (4.6) Fg. 4.3 Frst order dervatve estmaton. Estmated dervatve by the proposed method (full lne) for a varyng k and true dervatve (dashed lne) (p = 3) wth a Gaussan kernel to smooth the data. The bandwdths were selected va the procedure dscussed n Sect In the second smulaton, we llustrate the proposed estmator of the frst order dervatve when k s a functon of X, see(4.5). We consder the functon m(x) = X(1 X)sn((2.1π)/(X+0.05)), X U[0.25, 1], n = 1000 and e N(0, ). The densty was estmated usng kernel densty estmaton n combnaton wth the solve-the-equaton bandwdth selector. The value of k vared between 8 and 17. Fgure 4.3 shows the result. 4.5 Concluson We proposed a methodology to estmate frst order dervatves n the random desgn settng wthout estmatng the regresson functon va smoothng a weghted dffer-

8 38 K. De Brabanter and Y. Lu ence sequence. We derved L 2 rates and establshed consstency of the estmator. The newly created data sets are no longer ndependent and dentcally dstrbuted random varables. Therefore, we used bmodal kernels n the local polynomal regresson framework. Future research wll nclude the study of hgher order dervatves and behavor at the boundares n the random desgn settng. References 1. Chaudhur P, Marron JS (1999) SZer for exploraton of structures n curves. J Am Stat Assoc 94(447): De Brabanter K, De Brabanter J, Suykens JAK, De Moor B (2011) Kernel regresson n the presence of correlated errors. J Mach Learn Res 12: De Brabanter K, De Brabanter J, Gjbels I, De Moor B (2012) Dervatve estmaton wth local polynomal fttng. J Mach Learn Res 14: De Brabanter K, Gjbels I, Opsomer J (2014) Local polynomal regresson wth correlated errors n random desgn. Manuscrpt n preparaton 5. Fan J, Gjbels I (1996) Local polynomal modelng and ts applcatons. Chapman & Hall, London 6. Györf L, Kohler M, Krzyżak A, Walk H (2002) A dstrbuton-free theory of nonparametrc regresson. Sprnger, Berln 7. Müller H-G, Stadtmüller U, Schmtt T (1987) Bandwdth choce and confdence ntervals for dervatves of nosy data. Bometrka 74(4): Park C, Kang K-H (2008) SZer analyss for the comparson of regresson curves. Comput Stat Data Anal 52(8): Ramsay JO (1998) Dervatve estmaton. In: StatLb S-News, Thursday, 12 March, Avalable va Ramsay JO, Slverman BW (2002) Appled functonal data analyss. Sprnger, Berln 11. Rondonott V, Marron JS, Park C (2007) SZer for tme seres: a new approach to the analyss of trends. Electron J Stat 1: Sheather SJ, Jones MC (1991) A relable data-based bandwdth selecton method for kernel densty estmaton. J R Stat Soc, Ser B, Stat Methodol 53: Stone C (1985) Addtve regresson and other nonparametrc models. Ann Stat 13(2): Wahba G, Wang Y (1990) When s the optmal regularzaton parameter nsenstve to the choce of loss functon? Commun Stat, Theory Methods 19(5): Zhou S, Wolfe DA (2000) On dervatve estmaton n splne regresson. Stat Sn 10(1):93 108