5.1 Two-Step Conditional Density Estimator

Size: px

Start display at page:

Download "5.1 Two-Step Conditional Density Estimator"

Diane Campbell
6 years ago
Views:

1 5.1 Tw-Step Cditial Desity Estimatr We ca write y = g(x) + e where g(x) is the cditial mea fucti ad e is the regressi errr. Let f e (e j x) be the cditial desity f e give X = x: The the cditial desity f y is f (y j x) = f e (y g(x) j x) That is, we ca write the cditial desity f y i terms f the regressi fucti ad the cditial desity f the errr. This decmpsiti suggests a alterative tw-step estimatr f f: First, estimate g: Secd, estimate f e : The estimatr ^g(x) fr g ca be NW, WNW, r LL. The residuals are ^e i = y i ^g (X i ) : The secd step is a cditial desity estimatr (NW, WNW r LL) applied t the residuals ^e i as if they are bserved data. This gives a estimatr ^f e (e j x). The estimatr fr f is the ^f (y j x) = ^f e (y ^g(x) j x) The rst-rder asympttic distributi f ^f turs ut t be idetical t the ideal case where e i is directly bserved. This is because the rst step cditial mea estimatr ^g(x) cverges at a rate faster tha the secd step estimatr (at least if the rst step is de with a badwidth f the ptimal rder). e.g. if q = 1 the ^g(x) is ptimally cmputed with a badwidth h 1=5 ; s that ^g cverges at the rate 2=5 ; yet the estimatr ^f e cverges at the best rate 1=3 ; s the errr iduced by estimati f ^g is f lwer stchastic rder. The gai frm the tw-step estimatr is that the cditial desity f e typically has less depedece X tha the cditial desity f y itself. This is because the cditial mea g(x) has bee remved, leavig ly the higher-rder depedece. The accuracy f parametric estimati imprves as the estimated fucti becmes smther ad less depedet the cditiig variables. Partially this ccurs because reduced depedece allws fr larger badwidths, which reduces estimati variace.) As a extreme case, if f e (e j x) = f e (e j x) des t deped e f the X variables, the ^f e ca cverge at the 2=(q+4) rate f the cditial mea. I this case the tw-step estimatr actually has a imprved rate f cvergece relative t the cvetial estimatr. Tw-step estimatrs f this frm are fte emplyed i practical applicatis, but d t seem t have bee discussed much i the theretical literature. 51

2 We culd als csider a 3-step estimatr, based the expressis The 3-step estimatr is: y = g(x) + e e 2 = 2 (X) + j x f ( j x) f (y j x) = f y g(x) (x) j x First ^g(x) by parametric regressi, btai residuals ^e i : Secd, ^ 2 (x) by parametric regressi usig ^e 2 i as depedet variable. Obtai rescaled residuals ^ i = ^e i =^ (X i ) : Third, ^f ( j x) as the parametric cditial desity estimatr fr ^ i : The we ca set ^f (y j x) = ^f y ^g(x) ^(x) j x I cases f strg variace e ects (such as i acial data) this methd may be desireable. As the variace estimatr ^ 2 (x) cverges at the same rate as the mea ^g(x); the same rstrder prperties apply t the 3-step estimatr as t the 2-step estimatr. Namely, f shuld have reduced depedece x; s it shuld be relatively well estimated eve with large x-badwidths, resultig i reduced MSE relative t the 1-step ad 2-step estimatrs. Give these isights, it might seem sesible t apply the 2-step r 3-step idea t cditial distributi estimati. Ufrtuately the aalysis is t quite as simple. I this settig, the parametric cditial mea, cditial variace, ad cditial distributi estimatrs all cverge at the same rates. Thus the distributi f the estimate f the CDF f e i depeds the fact that it is a 2-step estimatr, ad it is t immediately bvius hw this a ects the asympttic distributi. I have t see a ivestigati f this issue. 52

3 6 Cditial Quatile Estimati 6.1 Quatiles Suppse Y is uivariate with distributi F: If F is ctiuus ad strictly icreasig the its iverse fucti is uiquely de ed. I this case the th quatile f Y is q = F 1 (): If F is t strictly icreasig the the iverse fucti is t well de ed ad thus quatiles are t uique but are iterval-valued. T allw fr this case it is cvetial t simply de e the quatile as the lwer bud f this edpit. Thus the geeral de iti f the th quatile is. q = if fy : F (y) g : Quatiles are fuctis frm prbabilities t the sample space, ad mtically icreasig i Multivariate quatiles are t well de ed. Thus quatiles are used fr uivariate ad cditial settigs. If yu kw a distributi fucti F the yu kw the quatile fucti q : If yu have a estimate ^F (y) f F (y) the yu ca de e the estimate ^q = if y : ^F (y) If ^F (y) is mtic i y the ^q will als be mtic i : Whe a smthed estimatr ^F (y) is used, the we ca write the quatile estimatr mre simply as ^q = ^F 1 (): Suppse that ^F (y) is the (usmthed) EDF frm a sample f size. I this case, ^q equals Y ([]) ; the [] th rder statistic frm the sample. If is t a iteger, [] is the greatest iteger less tha : We culd als view the iterval Y ([]) ; Y ([]+1) as the quatile estimate. We igre these distictis i practice. where Whe ^F (y) is the EDF we ca als write the quatile estimatr as ^q = argmi q X (Y i q) (u) = u [ 1 (u 0)] 8 >< (1 ) u u < 0 = >: u u 0 53

4 is called the check fucti. 6.2 Cditial Quatiles If the cditial distributi f Y give X = x is F (y j x) the the cditial quatile f Y give X = x is q = if fy : F (y j x) g = F 1 ( j x) Cditial quatiles are fuctis frm prbabilities t the sample space, fr a xed value f the cditiig variables. Oe methd fr parametric cditial quatile estimati is t ivert a estimated distributi fucti. Take a estimate ^F (y j x) f F (y j x) : The we ca de e ^q (x) = if y : ^F (y j x) Whe ^F (y j x) is smth i y we ca write this as ^q (x) = ^F 1 ( j x) : This methd is particularly apprpriate fr iversi f the smthed CDF estimatrs F ~ (y j x) : This iversi methd requires that ^F (y j x) be a distributi fucti (that it lies i [0; 1] ad is mtic), which is t esured if ^F (y j x) is cmputed by LL. The NW, WNW ad smthed versis are all apprpriate. Whe ^F (y j x) is a distributi fucti the ^q (x) will satisfy the prperties f a quatile fucti. 6.3 Check Fucti Apprach Ather estimati methd is t de e a weighted check fucti. either a lcally cstat r lcally liear speci cati. The lcally cstat (NW) methd uses the criteri This ca be de usig S (q j x) = X K H 1 (X i x) (Y i q) It is a lcally weighted the check fucti, fr bservatis clse t X i = x: The parametric quatile estimatr is The lcal liear (LL) criteri is ^q (x) = argmi S (q j x) : q S (q; j x) = X K H 1 (X i x) Y i q (X i x) 0 : 54

5 The estimatr is ^q (x) ; ^ (x) = argmi S (q; j x) : q; The cditial quatile estimatr is ^q (x) ; with derivative estimate ^ (x): Numerically, these prblems are idetical t weighted liear quatile regressi. 6.4 Asympttic Distributi The asympttic distributis f the quatile estimatrs are scaled versis f the asympttic distributis f the CDF estimatrs (see the Li-Racie text fr details). The CDF iversi methd ad the check fucti methd have the same asympttic distributis. The asympttic bias f the quatile estimatrs depeds whether a lcal cstat r lcal liear methd was used, ad whether smthig i the y directi is used. 6.5 Badwidth Selecti Optimal badwidth selecti fr parametric quatile regressi is less well studied tha the ther methds. As the asympttic distributis seem t be scaled versis f the CDF estimatrs, ad the quatile estimatr ca be viewed as a by-prduct f CDF estimati, it seems reasable t select badwidths by a methd ptimal fr CDF estimati, e.g. crss-validati fr cditial distributi fucti estimati. 55

ENGI 4421 Central Limit Theorem Page Central Limit Theorem [Navidi, section 4.11; Devore sections ]

ENGI 4421 Central Limit Theorem Page Central Limit Theorem [Navidi, section 4.11; Devore sections ] ENGI 441 Cetral Limit Therem Page 11-01 Cetral Limit Therem [Navidi, secti 4.11; Devre sectis 5.3-5.4] If X i is t rmally distributed, but E X i, V X i ad is large (apprximately 30 r mre), the, t a gd