R-Estimation in Linear Models with α-stable Errors

R-Estmaton n Lnar Modls wth α-stabl Errors Marc Halln Départmnt d Mathématqu and E.C.A.R.E.S., Unvrsté Lbr d Bruxlls London, Dcmbr 11, 2015 M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 1 / 48

Outln of th talk 1 Introducton : stabl dstrbutons 2 Lnar modls wth stabl nos 3 Rank tsts 4 R-stmaton M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 2 / 48

Introducton : stabl dstrbutons Structur 1 Introducton : stabl dstrbutons 2 Lnar modls wth stabl nos 3 Rank tsts 4 R-stmaton M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 3 / 48

Introducton : stabl dstrbutons Stabl dstrbutons Stabl dstrbutons ar xtrmly attractv from svral ponts of vw. Stochastc proprts (1) Stabl dstrbutons ar th only nondgnrat dstrbutons wth a doman of attracton : non-trval lmts of normalzd sums of ndpndnt dntcally dstrbutd trms ar ncssarly stabl. (2) Stabl famls ar qut flxbl : four paramtrs θ := (α, b, c, δ) Θ = (0, 2] [ 1, 1] R + R M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 4 / 48

Introducton : stabl dstrbutons (a) δ and c ar locaton-scal paramtrs : f (α,b,c,δ) (x) = f α,b,1,0 x δ c «/c. (b) α and b ar shap paramtrs : - α, th charactrstc xponnt s a tal ndx : th smallr α, th havr th tals - b s a skwnss paramtr : f s symmtrc f b = 0, totally skwd f b = 1. (3) som wll-known stabl dnsts (a) α = 2 (any b [ 1, 1]) : Gaussan dstrbuton, f (x) = 1 4π x2 /4. (b) α = 1 and b = 0 : Cauchy dstrbuton, f (x) = q (c) α = 1/2 and b = 1 : Lévy dstrbuton f (x) = 1. π(1+x 2 ) 1 2π 1/2x x 3/2. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 5 / 48

Introducton : stabl dstrbutons Stochastc modllng Emprcal vdnc of non-gaussan stabl bhavor s prsnt n a varty of flds, among whch conomcs, nsuranc, fnanc, sgnal procssng, tltraffc ngnrng,...... whr nglctng havy tals and asymmtry rsults n undrstmatd rsks, rcklss dcson makng, and qut svr losss. Studnt famls (gnrally wth thr dgrs of frdom or mor) thrfor ar qut popular n such aras but Studnt dstrbutons ar symmtrc, and Studnt tals wth thr or fv dgrs of frdom oftn ar stll too lght : Only stabl tals provd a rasonabl account for a numbr of stylzd facts Morovr,... M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 6 / 48

Introducton : stabl dstrbutons Statstcal nfrnc : Who s afrad of havy tals? Stabl famls : a statstcan s dram? Contrary to a wdsprad opnon, statstcal xprmnts nvolvng stabl nos ar xtrmly wll-bhavd. In ths talk, w concntrat on lnar (rgrsson) modls drvn by stabl rrors. W show blow that lnar modls wth..d. stabl rrors ar Locally Asymptotcally Normal (LAN, and vn ULAN, wth tradtonal root-n contguty rats) a most comfortabl stuaton, undr whch all nfrnc problms, n prncpl, can b solvd n a locally asymptotcally optmal way.... although, as a rul, th tradtonal Gaussan procdurs ar not vald anymor M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 7 / 48

Introducton : stabl dstrbutons Statstcal nfrnc : Who s afrad of havy tals? Stabl famls : a statstcan s nghtmar? No closd form for stabl dnsts!! (xcpt for th Gaussan, Cauchy and Lévy dnsts). No fnt momnts of ordr p for any p α! (xcpt for th Gaussan). No standard cntral-lmt bhavor of tradtonal (Gaussan) statstcs Hnc, 1 no closd-form lklhoods, vn lss for MLEs 2 no closd forms for optmal scors (log-drvatvs of th dnsts) 3 no closd forms for cntral squncs (n th LAN framwork)... M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 8 / 48

Introducton : stabl dstrbutons Statstcal nfrnc : Who s afrad of havy tals? A vry rch ltratur xsts on algorthmc mthods tryng to pallat th lack of xplct forms. That ltratur, t sms, rmans largly undrxplotd by practtonrs. Howvr, spcfyng or stmatng th tal paramtr α ramns dffcult/rsky assumng that approprat stabl-lklhood-basd" procdurs can b workd out, thy ar lkly to b snstv to volatons of th stablty assumpton : whl tradtonal Gaussan mthods notorously brak down undr stabl dnsts and nfnt varancs, th convrs s lkly to hold as wll : stabl lklhood-basd (stabl quas-lklhood) mthods ar lkly to run nto problms undr non-stabl condtons. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 9 / 48

Introducton : stabl dstrbutons Rank-basd nfrnc/rank tsts and R-stmaton Rank-basd mthods, thanks to dstrbuton-frnss, appar as a smpl and qut natural altrnatv to stabl quas-lklhood procdurs. Morovr, as w shall s, rank-basd mthods (n th contxt of lnar modls) achv paramtrc ffcncy at stabl rfrnc dnsts. Surprsngly, ranks sldom (nvr?) hav bn consdrd n th stabl contxt. Svral dlcat qustons ndd rman opn. Undr stabl dnsts or stabl nos, 1 whch rank tsts/r-stmators should w us? 2 what ar th prformancs of thos tsts?/ th asymptotc varancs of thos stmators? 3 fasablty? (computatonal problms n rlaton wth th absnc of xplct dnsts/scors... ) Thos ar th ssus w plan to nvstgat hr n th famlar contxt of lnar rgrsson. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 10 / 48

Lnar modls wth stabl nos Structur 1 Introducton : stabl dstrbutons 2 Lnar modls wth stabl nos 3 Rank tsts 4 R-stmaton M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 11 / 48

Lnar modls wth stabl nos Hypothss tstng n lnar modls wth stabl nos Dnot by H f () th hypothss undr whch th vctor of obsrvatons (X 1,..., X n ) satsfs th quaton whr - c := X = a + K l=1 c l l + ε, = 1,...,n, ( ) c 1,...,c K ar rgrsson constants, satsfyng th usual condtons ; - th ntrcpt a s a nusanc, th rgrsson paramtr = ( 1,..., K ) s th paramtr of ntrst ; - a + ε, = 1,...,n s a squnc of..d. random varabls wth dnsty f. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 12 / 48

Lnar modls wth stabl nos Undr H f (), th rsduals Z () = X ar..d. wth dnsty f. K k=1 c k k ( = 1,...,n) thn () If th rrors ar Gaussan, optmal tstng procdurs ar wll-known : optmal tsts ar basd on th Studnt statstc T n, whch s asymptotcally standard normal ; OLS stmators ar optmal. () If th rrors ar non-normal α-stabl, th optmal tstng/stmaton problm s a non-standard on, but LAN, as w shall s, n prncpl, provds asymptotcally optmal solutons. W concntrat on tstng null hypothss of th form = 0, but lnar rstrctons on could b consdrd as wll, ladng to th sam ffcncy conclusons and commnts. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 13 / 48

Lnar modls wth stabl nos ULAN for gnral lnar modl wth stabl rrors Th man thortcal tools throughout ar Local Asymptotc Normalty (LAN, actually ULAN) and L Cam s thrd Lmma. Lt c k := n 1 n =1 c k, c := (c 1,...,c K ), C := n 1 n and K := ( C ) 1/2. =1 c c, Assumpton (A1) For all n N, C s postv dfnt and convrgs, as n, to a postv dfnt K 2. Assumpton (A2) (Nothr condtons) For all k = 1,...,K, on has [ ] ( ) 2 / n ( ) 2 c tk c k c tk c k = 0. lm n max 1 t n t=1 M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 14 / 48

Lnar modls wth stabl nos Dnot by P θ, th probablty dstrbuton of X undr paramtr valus θ and. Lt Z () := X K k=1 c k k ( = 1,...,n), = 1,...,n b th rsdual assocatd wth. Undr P θ, a + ε : thy ar..d. wth dnsty f θ = f (α,b,c,a). Th followng thn holds., th rsduals Z () concd wth M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 15 / 48

Lnar modls wth stabl nos Thorm (ULAN) Suppos that (A1) and (A2) hold. Lt ν := n 1 2 K and fx θ = (α, b, c, a) Θ. Thn, th rgrsson modl wth stabl rrors s ULAN w.r.t.. Mor prcsly, for all R K, all squnc such that ν 1 ( ) = O(1) and all boundd squnc τ R K, [ () Λ := log dp θ,a, +ντ n θ, +ντ dp = t=1 log fθ θ,a, = τ θ ( ) 1 2 I(θ)τ τ + o P (1) undr H θ () as n, whr, sttng ϕ θ( ) := ḟθ( )/f θ ( ), I(θ) := (I(θ)I s th nformaton matrx) ( and () θ () = n 1/2 K ) n ( θ () a cntral squnc). =1 ϕ θ ϕ 2 θ (x)f θ(x)dx ( Z ) () c Z t (+ντ ) f θ Z t () L N(0, I(θ)I) (2.1) M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 16 / 48 ]

Lnar modls wth stabl nos Bnfts of ULAN Consquncs of ULAN ULAN allows us to 1 buld optmal paramtrc" and rank-basd tsts for H f θ () th valdty of th paramtrc" tsts rqurs corrct spcfcaton of f θ, and/or root-n-consstnt stmaton of θ, whl th valdty of rank-basd tsts dos not dpnd on th undrlyng dstrbuton ; ths ncluds, of cours, th stabl ons, and θ thus nds not b stmatd; 2 (va L Cam s thrd Lmma") comput th asymptotc local powrs of ths (and any othr) tsts (now, as a functon of th actual f, hnc, n th stabl cas, a functon of θ); 3 compar ths tsts through AREs ; 4 prform (on-stp) R-stmaton, wth th sam ARE valus as th corrspondng rank tsts. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 17 / 48

Rank tsts Structur 1 Introducton : stabl dstrbutons 2 Lnar modls wth stabl nos 3 Rank tsts 4 R-stmaton M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 18 / 48

Rank tsts Rank tsts Consdr a targt or rfrnc dstrbuton g at whch LAN also holds, wth cntral squnc g () = n 1 2 (K ) n ( ) ϕ g Z () c ; dfn th rank basd cntral squnc whr () = n 1 2 (K ) () : (0, 1) R : x ϕ g (G 1 (x)), =1 ( ) n R c, n + 1 () R = R ( 0 ) = (R 1,..., R n ) s th vctor of ranks of th rsduals Z 1 ( 0 ),..., Z n ( 0 ). =1 M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 19 / 48

Rank tsts Rank-basd tsts Proposton St T ( 0 ) = 1 () Thn T ( 0 ) s - asymptotcally ch-squar undr θ Θ 0 H θ ( 0 ), ( ) ( ) ( 0 ) ( 0 ) - asymptotcally ch-squar undr θ Θ 0 H θ ( 0 + n 1/2 τ) wth non-cntralty paramtr τ τ 2 (, θ), () wth (, θ) = 1 0 (u)ϕ θ(f 1 θ (u))du and () = 1 0 2 (u)du. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 20 / 48

Rank tsts Tsts and AREs Th corrspondng tsts (at nomnal asymptotc lvl α) consst n rjctng th null hypothss θ Θ 0 H θ ( 0 ) whnvr T ( 0 ) xcds th α-uppr quantl of th (cntral) ch-squar dstrbuton wth K dgrs of frdom. Lt and ( b two scor-gnratng functons, and dnot by ARE θ / ) th asymptotc rlatv ffcncy, undr stabl dnsty f θ, of th rank tst basd on ( 0 ) wth rspct to th rank tst basd on ( 0 ). Thn, T ARE θ T ( / ) = 2 (, θ) ( ) 2 (, θ) (). M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 21 / 48

Rank tsts Standard tsts... Standard tsts W appld thos rsults to th followng standard tsts : 1 van dr Wardn scors : (u) = Φ 1 (u), 2 Wlcoxon scors : 3 Laplac scors : (u) = π 3 (2u 1), (u) = 2sgn(F 1 (u)), wth F( ) cdf of standardzd doubl-xponntal. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 22 / 48

Rank tsts and lss standard ons... nw rank tsts basd on stabl scors"... but also to som non standard tsts, basd on stabl scors : 1 Cauchy scors : (u) = sn(2π(u 1/2)), 2 Lévy scors : (u) = 2 ( Φ 1 ((u + 1)/2) ) 2 (3 2 2 ( Φ 1 ((u + 1)/2 ) 2 ), 3 gnral stabl scors : (u) = ϕ f (F 1 (u)). M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 23 / 48

Rank tsts Rmarks on AREs Rcall that Thorm (Chrnoff-Savag, 1958) nf g ARE g(vdw/studnt) = 1 Thorm (Hodgs-Lhmann, 1956) nf g ARE g(w/studnt) = 0.864 In th prsnt contxt, howvr, snc Studnt tsts ar not vald, w rathr tak th van dr Wardn tsts (whch unformly domnat th Studnt ons) as a rfrnc for ARE computatons. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 24 / 48

Rank tsts AREs : Wlcoxon, Laplac, Cauchy vs van dr Wardn Fgur: AREs of Wlcoxon, Laplac and Cauchy wth rspct to van dr Wardn as functons of th tal ndx α, for varous valus of th skwnss paramtr b. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 25 / 48

Rank tsts A rmark on AREs Thorm supare g (W/vdW) = 6 g π 1.910 whr th suprmum s takn ovr all g wth fnt Fshr nformaton for locaton (whch ncluds stabl dnsts). Th suprmum s attand by th lmtng vrson of symmtrc havy tald laws wth nfntly fat tals,.g. α-stabls wth α 0 or Studnt t n wth n 0. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 26 / 48

Rank tsts AREs : Optmal stabl vs van dr Wardn α =1.6 α =1.7 α =1.8 α =1.9 b 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 0 0.2 0.4 α =1.6 0 1.2127 1.2045 1.1787 1.1332 1.1269 1.1075 1.0446 1.0407 1.0288 0.9444 0.9429 0.9386 0.2 1.2043 1.2129 1.2039 1.1277 1.1333 1.1256 1.0416 1.0447 1.0396 0.9433 0.9445 0.9428 0.4 1.1779 1.2033 1.2135 1.1100 1.12811 1.1337 1.0320 1.0425 1.0450 0.9396 0.9438 0.9450 α = 1.7 0 1.2010 1.1954 1.1772 1.1442 1.1394 1.1241 1.0756 1.0721 1.0615 0.9893 0.9879 0.9834 0.2 1.1942 1.2011 1.1962 1.1393 1.1444 1.1392 1.0727 1.0757 1.0714 0.9882 0.9894 0.9876 0.4 1.1731 1.1925 1.2017 1.1236 1.1387 1.1448 1.0631 1.0730 1.0759 0.9845 0.9886 0.9897 α =1.8 0 1.1645 1.1614 1.1511 1.1313 1.1284 1.1188 1.0878 1.0852 1.0771 1.0240 1.0226 1.0182 0.2 1.1600 1.1647 1.1628 1.1276 1.1315 1.1291 1.0852 1.0879 1.0852 1.0228 1.0240 1.0222 0.4 1.1465 1.1587 1.1654 1.1161 1.1267 1.1319 1.0767 1.0849 1.0881 1.0190 1.0231 1.0242 α = 1.9 0 1.1005 1.0994 1.0957 1.0878 1.0867 1.0831 1.0704 1.0693 1.0656 1.0405 1.0394 1.0361 0.2 1.0988 1.1008 1.1006 1.0862 1.0880 1.0876 1.0689 1.0706 1.0698 1.0394 1.0405 1.0394 0.4 1.0937 1.0988 1.1019 1.0813 1.0860 1.0887 1.0643 1.0686 1.0709 1.0360 1.0393 1.0406 Tabl: AREs for tsts basd on stabl scors wth rspct to van dr Wardn s. Rows corrspond to scors, columns to th (stabl) dnsts undr whch AREs ar computd. For nstanc, row 1 contans th AREs wth rspct to van dr Wardn of th tst basd on stabl scors for α = 1.6, b = 0, undr stabl dnsts wth tal paramtr α = 1.6 and skwnss b rangng from 0 through 0.4. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 27 / 48

Rank tsts Mont Carlo... W gnratd N = 2500 sampls from th rgrsson modls Y (l) = ((l/20))c + ǫ, = 1,...,n = 100, l = 0, 1, 2, 3, (3.2) whr th th ǫ s ar..d. wth cntrd alpha-stabl dstrbuton. Th rgrsson constants c ( = 1,...,100) (th sam ons across th 2500 rplcatons) wr drawn from th unform dstrbuton on [ 5, 5]. Obsrvatons Y (0) thus ar gnratd undr th null, Y (1), Y (2) and Y (3) undr ncrasng altrnatvs of th form = l/20, l = 1, 2 and 3. W prformd th varous tsts at nomnal lvl 5% for th null hypothss = 0. Crtcal valus wr computd from asymptotc dstrbutons. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 28 / 48

Rank tsts dnsty l dnsty l tst 0 1 2 3 0 1 2 3 φ vdw.0416.1728.3968.5824.0420.2324.6116.8728 φ W.0488.2600.5712.7724.0484.3176.7700.9564 φ L α =.5.0500.5992.9032.9780 α =.85.0476.4600.9084.9908 φ C = 0.0496.5304.8576.9500 = 0.0472.4304.8744.9740 φ 1.6;0.0532.2916.6180.8120.0516.3568.8224.9720 φt.0164.0244.0240.0204.0288.0344.0516.0872 φ vdw.0416.2004.4204.6164.0408.2484.6452.8836 φ W.0500.2784.5784.7752.0428.3388.7752.9580 φ L α =.5.0484.3236.6980.8812 α =.85.0472.3520.8136.9716 φ C =.4.0480.1856.3956.5480 =.4.0492.1932.5124.7632 φ 1.6;0.0508.3124.6444.8244.0476.3744.8300.9772 φt.0196.0224.0196.0212.0360.0420.0528.0764 φ vdw.0396.3472.6668.8176.0424.3488.8020.9604 φ W.0448.2732.5992.7684.0476.3216.7748.9524 φ L α =.5.0448.1028.2224.4036 α =.85 0496.1784.5248.8188 φ C =.99.0420.1880.2120.1776 =.99.0480.0500.0544.0780 φ 1.6;0.0428.2280.5396.7336.0488.2884.7444.9452 φt.0136.0224.0192.0252.0328.0376.0424.0688 Tabl: Rjcton frquncs (out of 2, 500 rplcatons), undr th null (l = 0) and undr altrnatvs (l = 1,2, 3), of th van dr Wardn tst φ vdw, th Wlcoxon tst φ W, th Laplac tst (th sgn tst) φ L, th Cauchy tst φ C, th tst φ 1.6;/0 (optmal at th stabl dstrbuton wth α = 1.6 and = 0) and th Studnt tst φ t. Undrlyng stabl dnsts wth α =.5 and.85. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 29 / 48

Rank tsts dnsty l dnsty l tst 0 1 2 3 0 1 2 3 φ vdw.0424.3964.9208.9968.0340.4488.9556.9996 φ W.0512.4580.9540.9992.0416.4848.9660.9996 φ L α = 1.6.0516.3724.9004.9972 α = 1.8.0428.3740.9028.9984 φ C = 0.0488.2788.7400.9624 = 0.0440.2392.7044.9520 φ 1.6;0.0580.4864.9624.9996.0432.4880.9680.9996 φt.0436.2700.6948.8700.0468.4052.8720.9692 φ vdw.0396.3972.9208.9988.0364.4436.9600 1.000 φ W.0440.4512.9548 1.000.0444.4860.9724 1.000 φ L α = 1.6.0492.3568.8952.9956 α = 1.8.0508.3832.9024 1.000 φ C =.4.0552.2164.6476.9228 =.4.0536.2120.6616.9312 φ 1.6;0.0460.4676.9628 1.000.0468.4944.9752 1.000 φt.0464.2836.6848.8748.0468.4064.8844.9664 φ vdw.0392.4404.9504.9992.0372.4408.9728 1.000 φ W.0492.4584.9532.9992.0400.4624.9768 1.000 φ L α = 1.6.0524.3332.8684.9948 α = 1.8.0480.3440.8976.9976 φ C =.99.0500.1352.4172.7512 =.99.0464.1736.5440.8708 φ 1.6;0.0552.4392.9472.9988.0408.4608.9676 1.000 φt.0440.2824.7120.8664.0496.3916.8800.9696 Tabl: Rjcton frquncs (out of 2, 500 rplcatons), undr th null (l = 0) and undr altrnatvs (l = 1,2, 3), of th van dr Wardn tst φ vdw, th Wlcoxon tst φ W, th Laplac tst (th sgn tst) φ L, th Cauchy tst φ C, th tst φ 1.6;/0 (optmal at th stabl dstrbuton wth α = 1.6 and = 0) and th Studnt tst φ t. Undrlyng stabl dnsts wth α = 1.6 and 1.8. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 30 / 48

Rank tsts alpha=.5 alpha=.85 Powr 0.0 0.4 0.8 Powr 0.0 0.4 0.8 0 5 10 15 20 l 0 5 10 15 20 l alpha=1.5 alpha=1.75 Powr 0.0 0.4 0.8 Powr 0.0 0.4 0.8 0 5 10 15 20 l 0 5 10 15 20 l alpha=1.9 alpha=2 Powr 0.0 0.4 0.8 Powr 0.0 0.4 0.8 0 5 10 15 20 0 5 10 15 20 Fgur: Powr curvs of th van dr Wardn (sold ln) and Studnt (dottd ln) tsts computd from 10,000 rplcatons for varous symmtrc stabl rrors. Sampl sz s n = 100 and rgrsson constants ar drawn from th unform dstrbuton on [ 5, 5]. l l M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 31 / 48

R-stmaton Structur 1 Introducton : stabl dstrbutons 2 Lnar modls wth stabl nos 3 Rank tsts 4 R-stmaton M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 32 / 48

R-stmaton Classcal stmaton mthods n th prsnc of havy tals fal to provd satsfactory solutons. (a) OLS stmators : consstncy rat dpnds on th tal ndx α (Samorodntsky t al. 2007) ; that rat s strctly lss than th optmal root-n rat. (b) Stabl MLEs : problm of th absnc of closd form lklhoods and th nformaton matrx, morovr, s not block-dagonal. (c) Lnar unbasd stmators : consstncy rats agan crucally dpnd on α and ar strctly lss than th optmal root-n ons ; asymptotc covarancs dpnd on α as wll. (d) LAD (Last Absolut Dvatons) stmators : achv rat-optmal consstncy at arbtrary stabl dnsts. But LAD stmators ar optmal undr (lght-tald) doubl-xponntal nos, and cannot b ffcnt undr any havy-tald stabl dnsts. Altrnatv : stmaton basd on ranks! M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 33 / 48

R-stmaton Hodgs-Lhmann R-stmaton Estmaton mthods basd on ranks n short, R-stmaton go back to Hodgs and Lhmann (1963) (on-sampl and two-sampl locaton modls, basd on th Wlcoxon and van dr Wardn (sgnd) rank statstc) ; xtnson to rgrsson was mad possbl by určková (1971) and Koul (1971) Undr classcal Argmn form, th Hodgs-Lhmann R-stmator HL of s dfnd as HL := argmn t R K Q (R (t)), whr Q (R ()) s a (sgnd)-rank tst statstc for th H 0 : = (two-sdd tst). M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 34 / 48

R-stmaton Man advantags of HL ovr usual M-stmators (undr paramtr valu and rror dnsty f, and standard root-n consstncy condtons) : n 1/2 ( HL ) s asymptotcally quvalnt to a functon whch dpnds on th unknown actual dnsty f but s masurabl wth rspct to th ranks R () of th unobsrvabl nos ( quvaranc" undr monoton contnuous transformatons of th rrors) th asymptotc rlatv ffcncs (AREs) of th R-stmator HL wth rspct to othr R-stmators, or wth rspct to ts Gaussan compttor (OLS or Gaussan MLE, whn root-n consstnt) ar th sam as th AREs of th corrspondng rank tsts wth rspct to thr Gaussan compttors M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 35 / 48

R-stmaton Man advantags of HL ovr usual M-stmators (undr paramtr valu and rror dnsty f, and standard root-n consstncy condtons) : n 1/2 ( HL ) s asymptotcally quvalnt to a functon whch dpnds on th unknown actual dnsty f but s masurabl wth rspct to th ranks R () of th unobsrvabl nos ( quvaranc" undr monoton contnuous transformatons of th rrors) th asymptotc rlatv ffcncs (AREs) of th R-stmator HL wth rspct to othr R-stmators, or wth rspct to ts Gaussan compttor (OLS or Gaussan MLE, whn root-n consstnt) ar th sam as th AREs of th corrspondng rank tsts wth rspct to thr Gaussan compttors Man dsadvantag of ϑ HL n th rgrsson contxt th Argmn bcoms rapdly mpractcal as th dmnson of ncrass (optmzaton ovr a K-dmnsonal grd) vn for small K, a grd mthod nvolvng stabl scors s computatonally nfasbl M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 35 / 48

R-stmaton On-stp R-stmators. W thrfor rathr rcommnd a lnarzd form of th dfnton of th form prlmnary root-n consstnt + rank basd mprovmnt. Hr th prlmnary root-n consstnt stmator wll b th LAD stmator (â of (a, ), obtand by mnmzng th L 1-objctv functon (â LAD, ˆ LAD) := argmn (a,) R K+1 nx =1 Z (). LAD, ˆ LAD ) For th rank-basd mprovmnt, consdr (agan) () := n 1 2 K nx =1! R c, (4.3) n + 1 whch satsfs an asymptotc lnarty proprty ( + ν τ ) () = (,g)τ + o P(1). M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 36 / 48

R-stmaton In prncpl, th on-stp R-stmator of should thn tak th vry smpl form := ˆ LAD + ν 1 (,g) (ˆ LAD) From th asymptotc lnarty of, w gt ν 1 ( ) s asymptotcally N(0,( ()/ 2 (,g))i K ) undr P g,a,. Ths n turn mpls that ν 1 ( ), for (u) = ϕ f (F 1 (u)), s asymptotcally N(0, 1 ()I K ) undr P f,a,,.. rachs paramtrc ffcncy at corrctly spcfd dnsty f = g. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 37 / 48

R-stmaton Unfortunatly, th scalar cross-nformaton quantty (,g) s not known. Thus, s not a gnun" stmator. That cross-nformaton quantty (, g) has to b consstntly stmatd. To obtan such a consstnt stmator, w adopt hr an da frst dvlopd n Halln, Oja and Pandavn (2006) and gnralzd n Cassart, Halln and Pandavn (2010). M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 38 / 48

R-stmaton Th on-stp R-stmator s such that := ( b 1 (,g)) = ˆ LAD + ν b 1 (, g) (ˆ LAD) n 1/2 ( ) s asymptotcally normal wth man zro and covaranc matrx ` ()/ 2 (,g) K 2 undr P g,a, wth g F lttng (u) = ϕ f (F 1 (u)), achvs th paramtrc ffcncy bound undr P f,a, th asymptotc rlatv ffcncs of R-stmators clarly concd wth thos of th corrspondng rank tsts M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 39 / 48

R-stmaton Tabl: AREs of R-stmators wth rspct to LAD stmators Estmators Undrlyng stabl dnsty α = 2; b = 0 α = 1.8 ; b = 0 α = 1.8 ; b = 0.5 α = 0.5 ; b = 0.5 /ˆ LAD W 1.4999 1.3888 1.3984 1.7776 /ˆ LAD vdw 1.5708 1.3056 1.3285 1.251 /ˆ LAD C 0.6759 0.7880 0.7769 2.007 /ˆ LAD 1.8;0 1.4459 1.4183 1.4222 1.6453 1.8;.5 /ˆ LAD 1.4452 1.3969 1.4459 1.4432.5;.5 /ˆ LAD 0.0925 0.1099 0.1175 21.2364 AREs for R-stmators basd on varous scors wth rspct to th LAD stmator. Columns corrspond to th (stabl) dnsts undr whch AREs ar computd, rows to th scors consdrd : Wlcoxon ( W), van dr Wardn ( vdw ), Cauchy ( C), and thr (δ = 0, γ = 1) stabl scors ( α;b ); rcall that th R-stmator basd on Laplac scors asymptotcally concds wth th LAD stmator. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 40 / 48

R-stmaton Wlcoxon/LAD Cauchy/LAD VdW/LAD ARE 0 1 2 3 4 b=0 b=0.2 b=0.4 b=0.6 b=0.8 ARE 0 1 2 3 4 ARE 0 1 2 3 4 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 Fgur: AREs of R-stmators basd on Wlcoxon, Cauchy and van dr Wardn scors, wth rspct to th LAD stmator, as a functon of α and for varous valus of b. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 41 / 48

R-stmaton alpha= 0.2 / LAD alpha= 0.4 / LAD alpha= 0.6 / LAD alpha= 0.8 / LAD 0 1 2 3 4 b=0 b=0.2 b=0.4 b=0.6 b=0.8 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 alpha= 1.2 / LAD alpha= 1.4 / LAD alpha= 1.6 / LAD alpha= 1.8 / LAD 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 0.5 1.0 1.5 2.0 Fgur: AREs undr stabl dstrbutons of R-stmators basd on varous stabl scors wth rspct to th LAD stmator, as a functons of α and b. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 42 / 48

R-stmaton W gnratd M = 1000 sampls from two multpl rgrsson modls, wth two rgrssors, and Y (1) = c 1 + c 2 + ǫ, = 1,..., n = 100, (4.4) Y (2) = c 1 + c 2 + c 3 + c 4 + ǫ, = 1,..., n = 100, (4.5) wth four rgrssors, both wth alpha-stabl..d. ǫ s. Th rgrsson constants c j (th sam ons across th 1000 rplcatons) wr drawn (ndpndntly) from th unform dstrbuton on [ 1, 1] 2 and [ 1, 1] 4, rspctvly. Lttng 1 K := (1,1,..., 1) R K, th tru valus of th rgrsson paramtrs ar thus = 1 2 n modl (4.4) and = 1 4 n modl (4.5). M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 43 / 48

R-stmaton Tabl: Emprcal bas and MSE for varous stmators of n (4.4) (2 rgrssors) Estmator ˆ LS ˆ LAD Undrlyng stabl dnsty (α/b) α = 2/b = 0 α = 1.8/b = 0 α = 1.8/b = 0.5 α = 1.2/b = 0 α = 1.2/b = 0.5 (Bas).00193 -.00134.01385.18680. -.19255 (MSE).06770.19459.27336 124.46 88.070 (Bas).00167 -.00087..00502.02995.00646 (MSE).10674.10411.11638.11560.13396 (Bas).00256 -.00136.00694.03376. -.00243 vdw (MSE).06878.07694..08545.15165..14499 (Bas).00076.00015.00920.02957 -.00147 W (MSE).07234.07454.08366.12060.12219 (Bas).00167 -.00087.00502.02995.00646 L (MSE).10674.10411..11638.11560.13396 (Bas).00250.00063.00883.03046.00068 1.8/0 (MSE).07088.07457..08310.12976..12820 (Bas).00187 -.00119.01057.03284 -.00037 1.8/.5 (MSE).07104.07683.08139.13562.12398 (Bas).00424.00353.01373.02155 -.00363 1.2/0 (MSE).11613.09812.11040.09641.10971 (Bas).00670 -.00418.01609.02735.00310 1.2/.5 (MSE).11416.10382.10822.11455.08917 (Bas).01070.03350.00357.04768 -.01671.5/.5 (MSE).22575.28311.24386.35926.18999 (Bas) -.01668 -.01040 -.00253.04306 -.01664 HL;vdW (MSE).07936.08958.09508.20227.20441 (Bas) -.00672 -.02019 -.01113 -.01052 -.03408 HL;W (MSE).08225.09071.09702.16290.14918 (Bas) -.02274 -.02834 -.01923 -.01504 -.05129 HL;1.8/0 (MSE).09066.10291.10488.18247.19072 Emprcal bas and MSE of th LSE ˆ, th LAD ˆ and varous rank-basd stmators computd from 1000 rplcatons LS LAD of modl (4.4) wth sampl sz n=100, undr varous stabl rror dstrbutons. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 44 / 48

R-stmaton Tabl: Emprcal bas and MSE for varous stmators of n modl (4.5) (4 rgrssors) Estmator ˆ LS ˆ LAD Undrlyng stabl dnsty (α/b) α = 2/b = 0 α = 1.8/b = 0 α = 1.8/b = 0.5 α = 1.2/b = 0 α = 1.2/b = 0.5 (Bas).00314.01367 -.01945-4.09468 -.09272 (MSE).06339.30161.12752 15818.91 39.45292 (Bas).00693.00880 -.00774 -.00652.00352 (MSE).09995.09992.09548.08495.09984 (Bas).00378.00638 -.01177 -.00763 -.01262 vdw (MSE).06463.06964.07238.11369.11015 (Bas).00542.00579 -.01236 -.00624 -.00774 W (MSE).06811.06847.06988.09038.09127 (Bas).00693.00880 -.00774 -.00652.00352 L (MSE).09995.09992.09548.08495.09984 (Bas).00499.00531 -.01221 -.00445 -.00980 1.8/0 (MSE).06755.06735.07021.09908.09562 (Bas).00339.00526 -.01109 -.00438 -.01151 1.8/.5 (MSE).06686.06914.06977.10095.09397 (Bas).00802.00608 -.01297.00682.00404 1.2/0 (MSE).10763.09229.08986.07061.08406 (Bas).00291.00024 -.01401.00396 -.00231 1.2/.5 (MSE).10332.09233.08567.09036.07037 (Bas).03400.03653 -.02823 -.05925 -.00469.5/.5 (MSE).30150.35030.28818.43049.18807 (Bas).00401.00634 -.01208 -.00704 -.01234 HL;vdW (MSE).06513.06968.07266.11310.10956 (Bas).00513.00623 -.01285 -.00547 -.00755 HL;W (MSE).06854.06855.07006.09010 0.09100 (Bas).00494.00582 -.01245 -.00396 -.01081 HL;1.8/0 (MSE).06783.06753.07037.09854.09594 Emprcal bas and MSE of th LSE ˆ, th LAD ˆ and varous rank-basd stmators computd from 1000 rplcatons LS LAD of modl (4.5) wth sampl sz n=100, undr varous stabl rror dstrbutons. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 45 / 48

R-stmaton Estmator Tabl: On-stp R-stmaton vrsus Argmn Undrlyng stabl dnsty (α/b) α = 2/b = 0 α = 1.8/b = 0 α = 1.8/b = 0.5 α = 1.2/b = 0 α = 1.2/b = 0.5 α = 0.5/b = K = 6 (Bas) -.01991 -.00485.01084 -.01890.02246.00162 vdw (MSE).07707.08821.08935.16485.15258.61554 (Bas) -.19519 -.19834 -.19202 -.36809 -.30435 -.59222 HL;vdW (MSE).24257.27483.27461.58981.52245 2.51344 K = 10 (Bas) -.00877.00607.00187 -.00807 -.01376.06003 vdw (MSE).07834.09133.08641.16835.15545 1.4346 (Bas) -.91080 -.89626 -.92196-1.00979 -.99976 -.97662 HL;vdW (MSE) 1.04321 1.07289 1.09949 1.50269 1.43327 3.23870 K = 15 (Bas) -.00374 -.01421 -.00575.02479 0.00271.01123 vdw (MSE).08894.10969.10539.20918.19621 2.00335 (Bas) -1.07573-1.11915-1.11057-1.23107-1.21492-1.31910 HL;vdW (MSE) 1.19685 1.33319 1.32890 1.91879 1.88120 4.32374 Emprcal bas and MSE of th on-stp and Argmn vrsons and of th van dr Wardn R-stmator vdw HL;vdW computd (va th Nldr-Mad (1965) algorthm for th Hodgs-Lhmann cas) from 1000 rplcatons of modl (4.5) wth K = 6, 10, 15, sampl sz n=100 and varous stabl rror dstrbutons. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 46 / 48

R-stmaton Conclusons 1 contrary to common blf, rgrsson xprmnts wth stabl rrors ar LAN, wth tradtonal root-n rat ; 2 tradtonal tstng/stmaton mthods, howvr, as a rul, ar rat-suboptmal n th prsnc of stabl rrors 3 On xcpton s th LAD stmator, along wth th rlatd mdan-tst or Laplac scor tsts whch ar rat-optmal but far from ffcnt 4 Rank-basd mthods allow for tstng and stmaton mthods that rman vald rrspctv of th tal ndx and skwnss paramtr... 5 but can b tund n ordr to rach paramtrc ffcncy at gvn stabl dstrbuton 6... and, for adquat scors, yld unformly bttr prformanc than LAD stmators and Laplac scor tsts ovr th class of stabl dnsts wth α 1 or α 1... 7 Fnally, th on-stp form of R-stmaton sgnfcantly outprforms th Argmn form n fnt sampl M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 47 / 48

R-stmaton Rfrncs Halln, M., Y. Swan, T. Vrdbout and D. Vrdas (2011) Rank-basd tstng n lnar modls wth stabl rrors. ournal of Nonparamtrc Statstcs, to appar. Halln, M., Y. Swan, T. Vrdbout and D. Vrdas (2011) On-stp R-stmaton n lnar modls wth stabl rrors. ournal of Economtrcs, to appar. M. Halln (ULB) R-Estmaton n Lnar Modls wth α-stabl Errors London, Dcmbr 11, 2015 48 / 48