Partial Linear Quantile Regression and Bootstrap Confidence Bands

Size: px

Start display at page:

Download "Partial Linear Quantile Regression and Bootstrap Confidence Bands"

Linda Park
5 years ago
Views:

Partial Linear Quantile Regressin and Btstrap Cnfidence Bands Wlfgang K. Härdle Ya acv Ritv Sng Sng Ladislaus vn Brtkiewicz Chair f Statistics C.A.S.E.

1 Partial Linear Quantile Regressin and Btstrap Cnfidence Bands Wlfgang K. Härdle Ya acv Ritv Sng Sng Ladislaus vn Brtkiewicz Chair f Statistics C.A.S.E. - Center fr Applied Statistics and Ecnmics Humbldt-Universität zu Berlin Department f Statistics The Hebrew University f Jerusalem

2 Risky t see nly part(s) f the truth!

3 $ N (age) lg(salary) Years Matthew Effect Yu et al. (2003)

LIDAR (Light Detectin and Ranging) 400 450 500 550 600 650 700 0.8 0.6 0.4 0.2 0.

4 LIDAR (Light Detectin and Ranging) Range Lgrati Lgrati (received light frm 2 laser surces) Range (distance befre the light is reflected back)

5 Mtivatin 1-4 Quantile Regressin QR: cnditinal behavir f a respnse Y Median regressin = mean regressin (symmetric) Gradually develping int a cmprehensive strategy fr cmpleting the regressin predictin, Kenker & Hallck (2001) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

6 Mtivatin 1-5 Example Financial Market & Ecnmetrics VaR (Value at Risk) tl t measure risk, Lauridsen (2000) Detect cnditinal heterscedasticity, Kenker & Bassett (1982) Labr Market Investigate differential effects, Buchinsky (1995) lg (Incme) = A(year, age, etc) +β B(educatin, gender, natinality, unin status, etc) + ε Inequality analysis, Juhn et al. (1993) Intersectin bunds, Chernzhukv et al. (2009) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

7 Mtivatin 1-6 Quantile Regressin l(x) = F 1 Y x (p) p-quantile regressin curve l(x) = linear (parametric) frm, Kenker & Bassett (1978) l h (x) quantile-smther Hw t decide between functinal frms? (glbal variability f the estimate, peak r valley really a feature?) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

8 Stchastic Integrals and Differential Equatins 5-2 Y X Mtivatin 1-7 This part f the curse prvides the tls fr the valuatin f Therem ptins. (Härdle and Sng (2009)) An apprximate We will define (1stchastic α) 100% prcesses cnfidence as slutins band ver f stchastic [0, 1] is differential equatins (SDE). l h (t) ± (nh) 1/2 {p(1 p)/ˆf X (t)} 1/2ˆf 1 {l h (t) t} Stchastic prcess {d n + inc(α)(2δ cntinuus lg n) time 1/2 } {λ(k)} 1/2, (1) A stchastic prcess in cntinuus time where c(α) = lg 2 lg lg(1 α) and ˆf is a cllectin X (t), ˆf f randm variables {X t ; t R + } with a cntinuus time{l variable h (t) t} are t. cnsistent estimates fr f X (t), f {l(t) t}. Emil Nrbert Julius Gumbel Wiener n BBI: Tl: Hungarian machine gun", x R 1 (KMT) SFE Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

Mtivatin 1-8 5 4 3 2 1 0-2 -1 0 1 2 Figure 1: The 0.9-quantile curve, the 0.9-quantile smther with h 0.

9 Mtivatin Figure 1: The 0.9-quantile curve, the 0.9-quantile smther with h 0.9 = 1.25 and 95% cnfidence bands. QRBSGumble Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

10 Mtivatin 1-9 Challenges L( l h - l ) (lg n) -1 exp{-exp(-x)} Gumbel, Emil J. 43 Emil Julius Gumbel BBI (lg n) -1 L * ( l * h - l g ) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

11 Mtivatin 1-10 Opprtunities Extend this t x R d and imprve band precisin? Hall (1991): btstrap imprvement (density) Hahn (1995): cnsistency f btstraping cdf Hrwitz (1998): btstrap (pintwise) fr median PLM: Green & Yandell (1985), Denby (1986), Speckman (1988) and Rbinsn (1988) Variable selectin fr QR: Liang and Li (2009) LASSO QR fr sparsity prblem, Bellni and Chernzhukv (2009) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

12 Outline 1. Mtivatin 2. Btstrap Cnfidence Bands 3. Btstrap Cnfidence Bands in PLMs 4. Mnte Carl Study 5. Labur Market Applicatins

13 Btstrap cnfidence bands 2-1 Quantile Regressin {(X i, Y i )} n i=1 i.i.d. rv s, x J = (a, b) fr sme 0 < a < b < 1, y R Suppse Y i = l(x i ) + ε i, ε i F ( X i ) with F (0 X i ) = p. Bth l & F are smth. Estimatr l h ( ): the slutin f n i=1 K h(x X i )1{Y i < l h (x)} n i=1 n i=1 K < p K h(x X i )1{Y i l h (x)} h(x X i ) n i=1 K h(x X i ) S n : any slwly varying functin (e.g., S 2 n = S n is valid... ). Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

14 Btstrap cnfidence bands 2-2 Lcal rate f cnvergence f l h δ n = h 2 + (nh) 1/2 = O(n 2/5 ) with h n = O(n 1/5 ) Auxiliary estimate l g with larger bandwidth g n = h n n ζ (ζ: 4/45) n j=1 ˆF ( X i ) = K h(x j X i )1{Y j l h (X i ) } n j=1 K h(x j X i ) Hw gd is this apprximatin? Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

15 Btstrap cnfidence bands 2-3 ˆF Apprximatin Perfrmance arund 0 Lemma [Franke and Mwita (2003), p14] If assumptins (A1, A2, A4) hld, then fr any small enugh (psitive) ε 0, sup ˆF (t X i ) F (t X i ) = O p (S n δ n ε 1/2 + ε 2 ). (2) t <ε,i=1,...,n,x i J Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

16 Btstrap cnfidence bands 2-4 The Btstrap Cuple U 1,..., U n : i.i.d. unifrm [0, 1] rv s Btstrap sample Y i = l g (X i ) + ˆF 1 (U i X i ), i = 1,..., n Cuple with the true cnditinal distributin: Y # i = l(x i ) + F 1 (U i X i ), i = 1,..., n. Given X 1,..., X n : Y 1,..., Y n and Y # 1,..., Y n # distributed. are equally Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

17 Btstrap cnfidence bands 2-5 A Very Clse Cuple Y i = l g (X i ) + ˆF 1 (U i X i ), i = 1,..., n Y # i = l(x i ) + F 1 (U i X i ), i = 1,..., n. Values f Y # i and Yi are meaningful nly if U i p < S n δ n. By the inverse functin therem arund p, we have i: Y # max Y # i i l(x i ) <S nδ n l(x i ) Y i + l g (X i ) = O p {S n δ 3/2 n }. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

18 Btstrap cnfidence bands 2-6 Hw Clse? q hi (Y 1,..., Y n ) def = l h (X i ) fr data set {(X i, Y i )} n i=1 Assumptin A3 gives: max l g (X i ) l g (X j ) l(x i ) + l(x j ) = O p (δ n ) X i X j <ch lh and l # h : lcal btstrap quantile and its cupled sample analgue. Then lh (X i) l g (X i ) = q hi [{Yj l g (X j ) + l g (X j ) l g (X i )}) n j=1] l # h (X i) l(x i ) = q hi [{Y # j l(x j ) + l(x j ) l(x i )} n j=1] Thus max lh (X i) l g (X i ) l # i h (X i) + l(x i ) = O p (δ n ). Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

19 Btstrap cnfidence bands 2-7 Btstrapping Apprximatin Rate Therem If assumptins (A1) (A3) hld, then sup lh (x) l g (x) l # x J h (x) + l(x) = O p(δ n ) = O p (n 2/5 ). Btstrap imprves the rate f cnvergence Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

20 Btstrap cnfidence bands 2-8 Why Oversmthing? Take care f bias with tuning parameter: g Härdle and Marrn (1991), let b h (x) ˆb h,g (x) def = E l # h (x) l(x) def = E lh (x) l g (x) [ } 2 ] Investigate E {ˆbh,g (x) b h (x) X1,..., X n. Hw fast it cnverges t 0? Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

21 Btstrap cnfidence bands 2-9 Oversmthing Therem Under sme assumptins, fr any x J [ } 2 ] E {ˆbh,g (x) b h (x) X1,..., X n h 4 {O p (g 4 ) + O p (n 1 g 5 )} T minimize RHS, g = O(n 1/9 ), g h, where h = O(n 1/5 ) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

22 Btstrap Cnfidence Bands in PLMs 3-1 The Multivariate Case x = (u, v) R d, v R: l(x) = u β + l(v) nte β als depends n p generally. Estimatin idea: ANOVA, apprximately linear frm (lcally) Partitin [0, 1] (fr v) in a n intervals I ni & regard l(v) as a cnstant item inside I ni. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

23 Btstrap Cnfidence Bands in PLMs 3-2 Tw Stage Estimatin Prcedure Linear quantile regressin inside each I ni + Weighted mean yields ˆβ: n a n ˆβ = arg min min ψ{y i β T U i l j 1 ( ) V i I ni } β l 1,...,l an i=1 j=1 Smth quantile estimate l h (v) frm (V i, Y i U i ˆβ) n i=1. Therem psitive definite matrices D, C, s.t. n( ˆβ β) L N{0, p(1 p)d 1 CD 1 } as n. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

24 Btstrap Cnfidence Bands in PLMs 3-3 Unifrm Cnsistency f ˆl h (v) Lemma Under assumptins (A7) & (A8), we have a.s. as n sup l h (v) l(v) C max{(nh/ lg n) 1/2, h α } (3) v J with cnstant C nt depending n n. If additinally α {lg( lg n) lg( nh)}/lg h, (3) can be further simplified t: sup v J l h (v) l(v) C{(nh/ lg n) 1/2 }. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

25 Btstrap Cnfidence Bands in PLMs 3-4 Multidimensinal Unifrm Cnfidence Bands Estimatin errr fr parametric part: O p (n 1/2 ). Btstrapping apprximatin errr fr nnparametric part: O p (n 2/5 ), dminating! Crllary Under the assumptins (A1) - (A8), an apprximate (1 α) 100% cnfidence band ver R d 1 [0, 1] is 1d u ˆβ + l h (v) ± [ˆf { l h (x) x} ˆf X (x)] α, where d α is based n the btstrap sample (defined later). Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

26 Mnte Carl study 4-1 Hw t Btstrap? 1) Simulate {(X i, Y i )} n i=1, n = 1000 w.r.t. f (x, y). f (x, y) = f y x (y sin x)1(x [0, 1]), (4) where f y x (x) is the pdf f N(0, x). 2) Cmpute l h (x) f Y 1,..., Y n and residuals ˆε i = Y i l h (X i ), i = 1,..., n. If we chse p = 0.9, then Φ 1 (p) = , l(x) = sin(x) x and the bandwidth is h = Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

27 Mnte Carl study 4-2 3) Cmpute the cnditinal edf: ˆF (t x) = n i=1 K h(x X i )1{ˆε i t} n i=1 K h(x X i ) with the quartic kernel K(u) = (1 u2 ) 2, ( u 1). 4) Generate rv ε i,b ˆF (t x), b = 1,..., B and cnstruct the btstrap sample Yi,b, i = 1,..., n, b = 1,..., B as fllws: with g = 0.2. Y i,b = l g (X i ) + ε i,b, Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

28 Mnte Carl study 4-3 5) Fr each btstrap sample {(X i, Yi,b )}n i=1, cmpute l h and the randm variable ] def d b = sup [ˆf {l h (x) x} ˆf X (x) lh (x) l g (x). (5) x J 6) Calculate the (1 α) quantile d α f d 1,..., d B. 7) Cnstruct the btstrap unifrm cnfidence band centered arund l h (x), i.e. l h (x) ± [ˆf {lh (x) x} ˆfX (x)] 1d α. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

29 Mnte Carl study % Quantile f Y Univariate X Figure 2: The real 0.9 quantile curve, 0.9 quantile estimate with crrespnding 95% unifrm cnfidence band frm asympttic thery and cnfidence band frm btstrapping. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

30 Mnte Carl study 4-5 Cnvergence Rate (n small) n Cv. Prb. Area (0.642) 0.58 (1.01) (0.742) 0.42 (0.58) (0.862) 0.31 (0.36) Table 1: Simulated cverage prbabilities & areas f nminal asympttic (btstrap) 95% cnfidence bands with 500 repetitin. T achieve same cv. prb., quantile regressin usually need mre bservatins than mean regressin Fr small n, btstrap s» asympttic s & nt sacrifice much n the band s width Use larger bandwidth n bth X & Y (1/ˆf {l h (x) x}) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

31 Mnte Carl study 4-6 PLM QR Bivariate data {(U i, V i, Y i )} n i=1, n = 8000 with: y = 2u + v 2 + ε Φ(p) (6) s.t. β = 2 even fr different p, where u [0, 2], v [0, 1] and ε is the standard nrmal rv. The real 0.9-quantile curve l(x) = 2u + v 2. h = 0.2 & g = 0.7. Fr the fllwing specific set f randm variables, a n = 20, ˆβ = Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

32 Mnte Carl study 4-7 # f Partitins? Beta Estimates Different Quantiles Beta Estimates Different Quantiles Beta Estimates Different Quantiles Figure 3: ˆβ with respect t different p fr different # f bservatins, i.e. n = 1000, n = 8000, n = Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

33 Mnte Carl study 4-8 a n n = 1000 n = 8000 n = n 1/3 / n 1/3 / n 1/3 / n 1/ n 1/ n 1/ n 1/ Table 2: SSE f ˆβ with respect t a n fr different numbers f bservatins. Suggest a n = n 1/3 (cst / perfrmance) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

34 Mnte Carl study 4-9 Nnparametric Part V Figure 4: Nnparametric part smthing, real 0.9 quantile curve with respect t v, 0.9 quantile smther with crrespnding 95% btstrap unifrm cnfidence band. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

35 Labr Market Applicatins 5-1 Labr Market Applicatin Hw incme depends n age w.r.t. different educatin levels? Relatin: lg (Wage) β Educatin + l(age) Administrative data frm the German Natinal Pensin Office West Germany, male, brn , sample 25-59, full-time, begin receiving a pensin in , uncensred n = bservatins Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

36 Labr Market Applicatins 5-2 German Pensin Office Data + Lnger perid + Large sample size + Precise measure f incme + Cmplete earning histry, true panel Rich peple nt recded (truncatin) Censring Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

37 Labr Market Applicatins 5-3 Educatin categries: lw educatin, apprenticeship and university Intuitin: E(y v, u = Lw educatin) < E(y v, u = Apprenticeship) < E(y v, u = University) Only part f the truth? Are the differences significant (unifrmly)? Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

38 Labr Market Applicatins 5-4 Bx Plt Lg Real Earnings Ages Figure 5: Bxplts fr lw educatin, apprenticeship & university grups crrespnding t different ages. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

39 Labr Market Applicatins lw educatin", 2 apprenticeship" and 3 university" /3 /2 = 25 partitins Quartic kernel, h = (after rescaling) Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

40 Labr Market Applicatins 5-6 β Estimates Beta Estimates Different Quantiles Figure 6: ˆβ crrespnding t 6, 13, 25 partitins. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

41 Labr Market Applicatins 5-7 Lwer End Lg Real Earnings Ages Figure 7: 95% unifrm cnfidence bands fr 0.20-quantile smthers with 3 different educatin levels lw educatin, apprenticeship & university. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

42 Labr Market Applicatins 5-8 Median Lg Real Earnings Ages Figure 8: 95% unifrm cnfidence bands fr 0.50-quantile smthers with 3 different educatin levels lw educatin, apprenticeship & university. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

43 Labr Market Applicatins 5-9 Upper End Lg Real Earnings Ages Figure 9: 95% unifrm cnfidence bands fr 0.80-quantile smthers with 3 different educatin levels lw educatin, apprenticeship & university Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

44 Labr Market Applicatins 5-10 Real effect f educatin fr incme? Only significant fr median & upper ends f the incme distributin Panel data, nt exactly i.i.d. (further research) Lngitudinal data, Yu, Sng and Härdle (2009) Time flies (technlgy level ), mre and mre high incme jbs require high educated peple. Time variatin f the ˆβ? Differential analysis t gender, labur unin status, natinalities etc? Inequality analysis t the (cnditinal) incme distributin?... Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

45 Quantnet - Open fr sharing

46 Labr Market Applicatins 5-12 Future Wrk VaR (99.9% quantile): U 1,..., U n imprtance sampling Key in prving btstrap rate: Mntne + Bunded influence! Extensin?... Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

47 Partial Linear Quantile Regressin and Btstrap Cnfidence Bands Wlfgang K. Härdle Ya acv Ritv Sng Sng Ladislaus vn Brtkiewicz Chair f Statistics C.A.S.E. - Center fr Applied Statistics and Ecnmics Humbldt-Universität zu Berlin Department f Statistics The Hebrew University f Jerusalem

48 Labr Market Applicatins 5-14 References A. Bellni and V. Chernzhukv Intersectin bunds: estimatin and inference CEMMAP Wrking Paper, 10/09, M. Buchinsky Quantile regressin, Bx-Cx transfrmatin mdel, and the U.S. wage structure, Jurnal f Ecnmetrics, 65: , V. Chernzhukv, S. Lee and A. M. Rsen L 1 -Penalized quantile regressin in high-dimensinal sparse mdels CEMMAP Wrking Paper, 19/09, Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

49 Labr Market Applicatins 5-15 References J. Franke and P. Mwita Nnparametric Estimates fr cnditinal quantiles f time series Reprt in Wirtschaftsmathematik 87, University f Kaiserslautern, P. Hall On cnvergence rates f suprema Prbab. Th. Rel. Fields, 89: , Jinyng Hahn Btstrapping Quantile Regressin Estimatrs Ecnmetric Thery, 11(1): , Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

50 Labr Market Applicatins 5-16 References W. Härdle and J. S. Marrn Btstrap simultaneus errr bars fr nnparametric regressin Ann. Statist., 19: W. Härdle and S. Sng Cnfidence bands in quantile regressin Ecnmetric Thery, frthcming, J. Hrwitz Btstrap methds fr median regressin mdels Ecnmetrica, 66: , Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

51 Labr Market Applicatins 5-17 References K. Jeng, W. Härdle and S. Sng A Cnsistent Nnparametric Test fr Causality in Quantile Revise and resubmit t Ecnmetric Thery, C. Juhn, K. M. Murphy and B. Pierce Wage Inequality and the Rise in Returns t Skill Jurnal f Plitical Ecnmy, 101(3): , R. Kenker and K. F. Hallck Quantile regressin Jurnal f Ecnmetric Perspectives, 15(4): , Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

52 Labr Market Applicatins 5-18 References J. Kmlós, P. Majr and G. Tusnády An Apprximatin f Partial Sums f Independent RV -s, and the Sample DF. I Z. Wahrscheinlichkeitstherie verw., 32: , K. Yu, Z. Lu and J. Stander Quantile regressin: applicatins and current research areas J. Ry. Statistical Sciety: Series D (The Statistician), 52: , Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

53 Appendix 6-1 Appendix - Assumptins λ i and C i : generic cnstants. A1. X 1,..., X n are an i.i.d. sample, and f X (x) λ 0. The quantile functin satisfies: l ( ) λ 1, l ( ) λ 2. A2. F (t x) have a density, f (t x) λ 3 > 0, cntinuus in x, and in t in the neighbrhd f 0. That is, fr sme A( ) and f 0 ( ) F (t x ) = p + f 0 (x)t + A(x)(x x) + R(t, x ; x), where sup t,x,x R(t,x ;x) t 2 + x x 2 <. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

54 Appendix 6-2 Nte that by Assumptin A1, l h (x) is the quantile f a discrete distributin. This distributin is equivalent t a sample size f O p (nh) frm a distributin with p-quantile whse biased is O p (h 2 ) relative t the true value. Let δ n be the lcal rate f cnvergence f the functin l h, essentially δ n = h 2 n + (nh n ) 1/2 = O(n 2/5 ), with h n = O(n 1/5 ). Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

55 Appendix 6-3 A3. The estimate l g, satisfies: sup l g (x) l (x) = Op(1), x J sup l g (x) l (x) = Op(δ n /h) x J (7) Nte that there is n S n term in (7) exactly because the bandwidth g n used t calculate l g is slightly larger than that used fr l h. As a result l g has a slightly wrse rate f cnvergence (as an estimatr f the quantile functin), but its derivatives cnverge faster. We assume: (A4). f X (x) is twice cntinuusly differentiable and f (t x) is unifrmly bunded in x and t by, say, λ 4. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

56 Appendix 6-4 (A7). The cnditinal densities f ( y), y R, are unifrmly lcal Lipschitz cntinuus f rder α (ull- α) n J, unifrmly in y R, with 0 < α 1, and (nh)/ lg n. (A8). inf v J ψ{y l(v) + ε}df (y v) q ε, fr ε δ1, where δ 1 and q are sme psitive cnstants, see als [?]. This assumptin is satisfied if there exists a cnstant q such that f {l(v) v} > q/p, x J. Partial Linear Quantile Regressin and Btstrap Cnfidence Bands

Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff

Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised