Kernel Adjusted Conditional Density Estimation

Transcription

1 Joural of Numerical Mathematics ad Stochastics, 10(1) : 0-31, JNM@S Euclidea Press, LLC Olie: ISSN Kerel Adjusted Coditioal Desity Estimatio Z.CHAKHCHOUKH, ad D. YAHIA Laboratory of Applied Mathematics, Biskra Uiversity, POBox 145, Algeria, yahia_dj@yahoo.fr Abstract. I this paper, we propose a ew kerel adjusted method i oparametric coditioal desity estimatio. Asymptotic properties of the ew estimator are determied ad the optimal choice of the smoothig parameters is preseted. I a simulatio study, the proposed method shows a good performace ad idicates that it is better tha existig comparable methods for all sample sizes. Key words : Coditioal Desity Estimatio, Kerel Estimatio, Mea Squared Error. AMS Subject Classificatios : 6G07, 6G0 1. Itroductio Let X i,y i i 1 be a bivariate radom variable (rv) with a commo probability desity fuctio f.,.. Our iterest i this paper is i the estimatio of the coditioal desity fuctio fy x of Y give X. This fuctio describes a comprehesive relatioship betwee resposes ad explaatory variables, ad ca be cosidered as geeral case of regressio. We ca give a simple defiitio to the coditioal desity fuctio as follows : fy x fx,y fx, for f xx 0. 1 where f. is a margial desity of X, ad coceive the coditioal desity fuctio as a source for various statistical quatities such as : mea, predictio iterval, momets, distributio, quatile ad so o. The coditioal desity estimatio was itroduced by Roseblatt i [1], ad a bias correctio was proposed by Hydma et al. i [8]. Fa et al. proposed i [4] a direct estimator based o local polyomial estimatio. The atural kerel estimator of fy x,see [15], is f y x f x,y, f x 0

2 1 Z. CHAKHCHOUKH ad D. YAHIA where f x,y 1 K h x X i H h y Y i i 1 is the kerel estimator of fx,y, ad f x 1 K h x X i i 1 is the kerel estimator of fx ad K h. : K./h, H h. : H./h, wherek. ad H. are kow as Tow kerel fuctios (usually a bouded ad symmetric pdf). Some examples of very commo kerel fuctios are the Epaechikov ad the Gaussia kerel. The parameter h : h is called the smoothig parameter, or the badwidth, ad it cotrols the smoothess of the resultig estimatio. I practice, the value of h depeds o the sample size ad satisfies the coditio h 0adh as. From the pricipal properties of estimator () that are usually studied, we metio oly some of them. We start with recallig that, uder stadard regularity coditios, the coditioal desity estimator has the asymptotic bias ad variace: biasfy x h K f x fx ad varfy x fy x x fy x x fy x y O h 4 O1/h, 4 K fy x K hfy x O1/h, fx where K : K tdt ad K : t Ktdt. Addig the squared bias (4) to the variace (5) gives the asymptotic mea square error AMSE : AMSEf y x : bias f y x varf y x. The optimal choice of h miimizig the last expressio satisfies h opt c 1/6 for c 0. The most importat of the previous facts is how to get a h opt, or i other words how to get a appropriate c. There are some methods as to how to obtai the value of c. The first possibility cosists i gettig the estimator of higher order derivatives of fx ad fy x by puttig the value subjectively prelimiary of h. The secod possibility is to apply the method of cross-validatio. It should be oted, i this respect, that Loader gave i [9] a brief discussio about cross-validatio. The previous two methods are cosidered as solutios to the problem of searchig for a suitable value for h, but they cotai some shortcomigs. I the first possibility, the estimator is ustable whe the sample size is small or moderate (see, [5]). With all these difficulties, Srihera ad Stute proposed ad studied i [14] a ew methodology o desity estimatio, to exted i approach of regressio, i aim to avoid the higher order derivatives. Their methodology relies o the choice of h ad also o the choice of K which 3 5

3 Kerel Adjusted Coditioal Desity Estimatio plays a very importat role i the efficiecy of estimator whe the kerel fuctio satisfies the sufficiet usual coditios. More precisely, this methodology of desity estimatio, [14], [3], was based o classical kerel desity that will become a fuctio of locatio scale family. I this paper, we develop a ew kerel type estimator of the coditioal desity fuctio, i which the kerel is adapted to the data but ot fixed. The ew method loads to a adaptive choice of the smoothig parameters. The basic techique of costructio of the proposed estimator is kid of a locatio-scale trasformatio, with reduced bias ad variace, which produces good results i all studied situatios. I Sectio, the proposed estimator is reported ad its bias, variace, mea squared error ad distributio behavior are determied ad preseted. I Sectio 3, simulatio studies are udertake to test the performace of the proposed estimator, ad compare it with the usual coditioal desity estimator.. Mai Results.1. The proposed estimator We follow the same methodology as i [14] ad [3], but specialize i this study i the estimatio of the coditioal desity fuctio. Startig from the formula (), we give a ew defiitio of the coditioal desity estimator. The procedure followed is to make a chage o the kerel fuctio; let K 0 be a kerel from the locatio-scale family associated with the margial desity f., that is K 0 t : K 0 t,, ft. 6 The scalig factor gives us more flexibility ad the choice of the adjusted kerel K 0 is based o the miimizatio of the AMSE. Sice the desity f i (6) is ot available, we have to replace it by the usual estimator f from above. Therefore, the classical desity estimator (3) becomes (see, [4]) f x h i 1 f i 1 x X i h K j 1 x X i h hx j. The choice of h, ad will be discussed later. Next, we ivoke formulas (1) ad () to obtai our ew estimator as follows f y x i 1 j 1 i 1 f x H h y Y i f x j 1

4 3 Z. CHAKHCHOUKH ad D. YAHIA i 1 K j 1 i 1 x X i h hx j H h y Y i K j 1 x X i h hx j. 7.. Asymptotic properties As usual, i order to study the asymptotic properties (i.e., bias, variace ad AMSE) ofthe give estimator, we apply the followig regularity assumptios: A 1 : EY ad EX. A : X has a desity f, which is cotiuously differetiable i a eighborhood of x. A 3 : fy x is twice cotiuously differetiable i a eighborhood of x. A 4 : K is a probability desity satisfyig K u Ku for all u. Furthermore, K has a fiite third momets : u 3 kudu. A 5 : h 0ad as. Theorem.1. Uder A 1 A 5, we have bias f y x f x f y x fx f y x varx O h 3 fx : Bx O h 3, 8 ad var f y x 1 f y x K f udu O1/h f x : 1 x O1/h, 9 where K is the same as i (5). Accordigly, the asymptotic mea squared error AMSE is AMSE f y x h 4 B x 1 x. Thus, the estimator is cosistet provided h 0 ad, as. Proof. Firstly, we cosider the bias part. Usig formula (15), we may write

5 Kerel Adjusted Coditioal Desity Estimatio 4 I : i 1 1 f x W xfy X i fy x j 1 i 1 K j 1 x X i h hx j fy X i fy x. We kow that f coverge to f i probability (see, [14]). Moreover, the umerator is equivalet to 1 K i j x X i h hx j fy z fy x. 10 The i the followig, we may cosider ay fixed i (10) rather tha. The expectatio becomes K Kv f fy f x z h hy x h vh hy x h vh hy fy z fy xfyfzdzdy fy x dydv. By Taylor s expasio, the last itegral, up to a Oh 3 term, equals : Kv f y/x h v hy 1 f y x h v hy fy fx f x Kvf y x Kvf y x Kvf y x 1 Kvf y x 1 Kvf y xf x h v hy h v hy h v hy h v hy 1 Kvf y xfyf x h v hy dydv 1 f x fyf x f xfy h v hy h v hy fyfxdvdy h v hy h v hy h v hy fydvdy dydv. dydv dydv dydv For EX ad ukudu 0, the first itegral is eglected. If we take 1 i 1 X i,

6 5 Z. CHAKHCHOUKH ad D. YAHIA the first, third, fifth ad sixth itegrals are O p h 3, as it is kow, h 1/6.Soweareleft with the calculatio of the secod ad the fourth itegrals. The secod itegral, at EX, equals, f x f y x varx oh 3. For, it equals : f x f y x y fdy Oh 3. As for the fourth itegral, we obtai the expasio f x f y x var X Oh 3. To coclude, whe the variace of I is o, we get the bias term (8), i.e. bias f y x f x f y x f x f y x var X Oh 3. f x Secodly, the variace part (9) is determied as follow. For each h 0 ad every 0, from (15), we have I 1 : i 1 i 1 j 1 W xh h y Y i fy X i. The predictable quadratic variatio of I 1 is give as W i x 1 X i 1 f x K i 1 h 4 i 1 j 1 1 X i x X i h hx j f f i probability, see here also [14]. Uder the assumptio A 5, the umerator takes the value 1 h 4 1 Xi K i j k x X i h hx j K x X i h hx k, 11 with EX. This is the U-statistic of degree three with a kerel depedig o h, ad. Its predictio equals h 4 1 y K K x y h hz x y h hu The last term is equivalet to fyfzfudydzdu..

7 Kerel Adjusted Coditioal Desity Estimatio 6 1 f y x K f udu O1/h. f x This completes the proof. The followig result cocers the distributio covergece of our estimator f y x. Theorem.. Uder the assumptios of theorem.1, if EX ad h o 1/6, the 1/ f y x fy x N0, x i distributio, where as before f y x x K f udu. f x Proof. Note that, by [14], f coverge to f i probability. Therefore, the study of the variace ad the distributio behavior of f y x, is based o that of the joit desity fuctio fy,x. Hece we itroduce its Hájek projectio (see, e.g. [6]) f 0 y,x, defied by h 3 i1 K K K x X i h hz Hy sfz,sdzds x t h hx i Hy sft,sdtds x t h hz Hy sfz,tftfsdzdtds. Note that f 0 y,x is a sum of i. i. d. rv s with E f 0 y,x E f y,x. Its variace equals Var f 0 y,x h 6 K x X i h hz K K x t h hx i fz,tftfsdzdtds x t h hz. Hy sfz,sdzds Hy sft,sdtds Hy s The it follows from argumets similar to those used i the proof of Lemma 4.1. i [3], that Var f 0 y,x fy x f x K f udu o1. Uder the assumptio of theorem. ad by defiitio of f y,x ad f 0 y,x, wehave f y,x f 0 y,x K x X i h hx j Hy Y 1 i i j 1

8 7 Z. CHAKHCHOUKH ad D. YAHIA K K K : x X i h hz x t h hx j x t h hs 1 L h X i,x j,y i. i j Hy Y i fx,zdxdz Hy Y i fx,tdxdt Hy Y i fyfsftdydsdt It is readily see that the last sum is a degeerate U-statistic of degree two, i.e., EL h X i,x j,y i L h X k,x j,y k X j 0, ad EL h X i,x j,y i L h X i,x k,y i X i 0, for i j k. As a coclusio we get E f y/x f y/x 1 h EL 3 X 1,X,Y 1. Note that, each terms i L h admits a secod momet of the order O. The E f y,x f y,x O 1 h o1. 13 The covergece of f y,x to fy,x follow immediately from (1) ad (13). Also we ca verify Lideberg s coditio (see, Corollary 1 i [1] by Brow ad Eagleso). Cosiderig the variace term i (15), the variace of (11) teds to zero as, ad the predictable quadratic variatio of I 1 equals 1 f y x f x K f udu o 1 : x o 1 h. The 1/ I 1 N0, x i distributio as. Here the proof completes..3. Optimal choice of smoothig parameters I the classical kerel estimatio literature, the optimal badwidth h opt ca be derived by miimizig the AMSE expressio. This optimal badwidth gives a trade-off betwee bias ad variace (e.g.). I the case of adjusted kerel estimatio, the AMSE depeds o h ad oly. Therefore, the ew method loads to a adaptive choice of both smoothig parameters h ad. Firstly, the optimal badwidth h opt ca be derived by differetiatig the AMSE of f with respect to h ad settig the derivatives to 0. Takig these derivatives ad simplifyig we obtai the followig expressio : h opt 3 x B x 1/6 : c 1/6. Next, we ote that, if h o 1/6, the bias is egligible. I such a situatio, the scale

9 Kerel Adjusted Coditioal Desity Estimatio 8 parameter should be chose as small as possible to make x also small. Therefore, the optimal choice of miimizig the AMSE f may be achieved by settig h 1/6, the 1/5 f x f y x f x f y xvarx opt fy x K f u du. 14 The differece betwee this classical methodology ad the ew methodology becomes clear. The classical kerel focuses o h 1 opt, whereas the ew methodology gives the pricipal role to the scale parameter. Note also that our estimator ca be rewritte as follow f y x : i 1 j 1 where W x : W ij x,ad W ij x K W xh h y Y i, x X i h hx j i 1 K j 1 x X i h hx j are called the weights fuctios. They are o egative ad satisfy : i 1 j 1 W x 1, for all x. As a result we ed up with the expressio f y x fy x WxH h y Y i fy X i i 1 j 1 Wxfy X i fy x, i 1 j 1 : I 1 I. 15 To fid a adaptive choice of, we do t focus o the aalytic form of Bx ad x i (8) ad (9) respectively. As I 1 ad Y i fy X i are coditioally cetred, ad sice 1 X i is ukow, we give W x 1 X i a predictable quadratic variatio of I 1, so as to i1 j1 replace it by Y i fy X i with h 1/6. As a coclusio, we ca say that, the bias term of is bias i 1 Thevariaceof is foud by W xf y X i fy x. j 1 1

10 9 Z. CHAKHCHOUKH ad D. YAHIA var i 1 W xy i fy X i. j 1 As a result, we obtai the adaptive scale parameter by miimizig the AMSE expressio : AMSE bias var. 3. Simulatio Study I this sectio we preset a simulatio study to illustrate the performace of our ew estimator f, give by Eq. (7), to establish the appropriate choice of the scale parameter. We will coduct the study by takig the data from the simple model Y j X j j, j 1,...,, where X j ad j are i. i.d. stadard ormal rv s. We will clarify the performace of our estimator f by assurig the preset results by usig the Mote Carlo simulatio, for small ad moderate sample sizes i certai poits of x. The coditioal desity fuctio of Y give X is give by fy x 1 4 exp 1 1 x xy1 y x. 4 Table 1: Bias, Var ad AMSE values over the boudary regio. f x Bias Var AMSE Bias Var AMSE f

11 Kerel Adjusted Coditioal Desity Estimatio 30 We measure the performace of the estimators by the asymptotic mea squared error (AMSE). The simulatio is based o 1000 replicatios, i each replicatio the sample sizes is: 5,50,100 ad 00 was used. For the kerel, we choose a Gaussia kerel. The choice of badwidth is very importat for a good performace of ay kerel estimator. I all cases, we cosider the asymptotic optimal global badwidth h opt 1/6, X ad the optimal scale parameter opt is as defied i Eq.(14). For each value of x 0.001, 0.05, 0. we have calculated the absolute bias Bias, variace Var ad the AMSE values of the two cosidered estimators ad have displayed the results i Table 1. The compariso shows that the proposed estimator exhibits the best performace. This is due to the fact that it is locally adaptive ad produces good results i all studied situatios. The mai results of our simulatio studies is that the proposed estimator ca be recommeded for bias ad variace reductio ad for improved boudary effects. Apparetly the overall f is the best choice amog the two estimators cosidered. The usual estimator f is clearly the worst, ad this is udoubtedly due to the boudary effect. Ackowledgmets The authors would like to thak a aoymous referee for his critical readig of the origial typescript. Refereces [1] B. M. Brow, ad G. K. Eagleso, Martigale covergece to ifiitely divisible laws with fiite variaces, Trasactios of the America Mathematical Society 16, (1971), [] A. Das Gupta, Asymptotic Theory of Statistics ad Probability, Spriger Verlag, Berli, 008. [3] G. Eicher,ad W. Stute, Kerel adjusted oparametric regressio, Joural of Statistical Plaig ad Iferece 14(9), (01), [4] J. Fa, Y. Qiwei, ad T. Howell, Estimatio of coditioal desities ad sesitivity measures i oliear dyamical systems, Biometrika 83, (1996), [5] W. Feluch, ad J. Koroacki, A ote o modified cross-validatio i desity estimatio, Computatioal Statistics & Data Aalysis 13, (199), [6] J. Hajek, Asymptotic ormality of simple liear rak statistics uder alteratives, The Aals of Mathematical Statistics 39, (1968), [7] W. Hardle, Applied Noparametric Regressio, Cambridge Uiversity Press, Cambridge, [8] R. J. Hydma, D. M. Bashtayk, ad G. K. Gruwald, Estimatig ad visualizig

12 31 Z. CHAKHCHOUKH ad D. YAHIA coditioal desities, Joural of Computatioal ad Graphical Statistics 5, (1996), [9] C. Loader, Local Regressio ad Likelihood, Spriger, New York,1999. [10] E. A. Nadaraya, Some ew estimates for distributio fuctios, Theory of Probability & Its Applicatios 9(3), (1964), [11] E. Parze, O estimatio of a probability desity fuctio ad mode, The Aals of Mathematical Statistics 33(3), (196), [1] M. Roseblatt, Remarks o some oparametric estimates of a desity fuctio, The Aals of Mathematical Statistics 7(3), (1956), [13] B.W. Silverma, Desity Estimatio for Statistics ad Data Aalysis, Chapma ad Hall, Lodo,1986. [14] R. Srihera, ad W. Stute, Kerel adjusted desity estimatio, Statistics & Probability Letters 81,(011), [15] G. R.Terrell, ad D. W. Scott, Variable kerel desity estimatio, Aals of Statistics 0(3), (199), [16] M. P. Wad, ad M. C. Joes, Kerel Smoothig, Chapma ad Hall, Lodo, [17] G. S.Watso, Smooth regressio aalysis, Sakhya, Series A 6, (1964), Article history: Submitted December, 08, 017; Revised May, 15, 018; Accepted September, 01, 018.