Inference based on robust estimators Part 2
|
|
|
- Herbert Bishop
- 5 years ago
- Views:
Transcription
1 Inference based on robust estimators Part 2 Matias Salibian-Barrera 1 Department of Statistics University of British Columbia ECARES - Dec 2007 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
2 General approach Fixed point equations ˆθ n = g n (ˆθ n ) Bootstrap the equations at the full-data estimator θ n = g n(ˆθ n ) Fast (e.g. weighted mean, weighted least squares) Underestimate variability (weights are not recomputed) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
3 General approach ˆθ n = g n (ˆθ n ) = g n (θ) + g n (θ) (ˆθ n θ) + R n n(ˆθ n θ) = [I g n (θ)] 1 n (g n (θ) θ) + o p (1) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
4 n (g n(ˆθ n ) ˆθ n ) n (g n(θ) θ) n (g n (θ) θ) n(ˆθn θ) [I g n (θ)] 1 n (g n(ˆθ n ) ˆθ ) n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
5 n(ˆθ n θ n ) n(ˆθ n θ) [I g n (θ)] 1 n (g n(ˆθ n ) ˆθ ) n ˆθ R n ˆθ [ ] 1 n = I g n (ˆθ n ) (g n(ˆθ n ) ˆθ ) n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
6 Applications Linear regression Standard errors (S-B and Zamar, 2002) Tests of hypotheses (S-B, 2005) Model selection (S-B and van Aelst, 2007) Multivariate location / scatter PCA (S-B, van Aelst, and Willems, 2006) Discriminant analysis (S-B, van Aelst, and Willems, 2007) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
7 Model selection Linear regression (y 1, x 1 ),..., (y n, x n ) Let α denote a subset of p α indices from {1, 2,..., p} y i = x αi β α + σ α ɛ αi i = 1,..., n, Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
8 all models α A are submodels of a full model ˆσ n S-scale estimate of full model For each model α A, the regression estimator ˆβ α,n solves 1 n n i=1 ρ 1 ( ) yi x αi ˆβα,n x i = 0. ˆσ n expected prediction error (conditional on the observed data) [ n ( M pe (α) = σ2 n E z i x αi ρ ˆβ ) ] α σ y, X, i=1 where z = (z 1,..., z n ) are future responses at X, independent of y, Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
9 Goodness of fit [ n ( σ 2 n E y i x ρ ˆβ )] αi α. σ i=1 parsimonious models are preferred Müller and Welsh (2005) { [ n ( M ppe (α) = σ2 y i x E ρ ˆβ )] } αi α + δ(n) p α + M pe (α), n σ i=1 where δ(n) δ(n)/n 0 (δ(n) = log(n)) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
10 Criteria [ n ( Mm,n(α) pe = ˆσ2 n n E yi x ˆβ ) ] αi α,n ρ ˆσ n y, X, i=1 { n ( Mm,n(α) ppe = ˆσ2 n yi x ˆβ ) } αi α,n ρ + δ(n) p α n ˆσ n E is the bootstrap mean select α A such that i=1 ˆα pe m, n = arg min α A Mpe m,n(α), + M pe m,n(α), ˆα ppe m, n = arg min α A Mppe m,n(α). Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
11 A c A such that β α contain all non-zero components of β In what follows we will assume that A c is not empty. The smallest model in A c will be true model α 0 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
12 Theorem Assume that (A1) n 1 x α i x α i Γ α > 0, n 1 ω α i x α i x α i Γ ω α > 0, and n 1 x α i 4 <, (A2) δ(n) = o(n/m) and m = o(n); (A3) n i=1 ρ 1 (r i(ˆβ α,n )/ˆσ n )x αi = 0, (A4) ˆσ n σ = O p (1/ n), ˆβ α,n β α = O p (1/ n); (A5) ρ 1 and ρ 1 are uniformly continuous, var(ρ 1 (ɛ α 0 )) <, var(ρ 1 (ɛ α 0 )) < and E(ρ 1 (ɛ α 0 )) > 0; and (A6) for any α / A c, var(ρ 1 (ɛ α)) < and with probability one lim inf n 1 n n i=1 1 ρ 1 (r i (ˆβ α )/ˆσ n ) > lim n n n ρ 1 (r i (ˆβ α0,n)/ˆσ n ). i=1 Then lim n P(ˆαppe m,n = α 0 ) = lim n P(ˆα pe m,n = α 0 ) = 1. Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
13 Example Los Angeles Ozone Pollution Data 366 daily observations on 9 variables Full model includes all second order interactions p = 45 Computational complexity Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
14 Example Backward elimination Starting from the full model Select the size-(k 1) model with best selection criteria Iterate Reduces search from 2 p to p(p + 1)/2 models Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
15 Using min α A M pe m,n(α) p = 6 Using min α A M ppe m,n(α) p = 7 Full model p = 45 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
16 Prediction error 5-fold CV trimmed (γ) prediction error estimators Full model p = 10 p = 7 p = 45 γ TMSE ρ TMSE ρ TMSE ρ ˆα pe m,n ˆα ppe m,n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
17 Diagnostic plots Standardized residuals Standardized residuals Fitted Values Fitted Values Standardized residuals Fitted Values Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
18 Average time (CPU seconds) to bootstrap an MM-regression estimator 1000 times on samples of size 200 p FRB CB Full model selection analysis on the Ozone dataset (p = 45) is reduced from 15 days (360 hours) to 4 hours. Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
19 Discriminant Analysis (Randles et al. 1978; Hawkins and McLachlan, 1997; Croux and Dehon, 2001; Hubert and Van Driessen, 2004) Populations π 1, π 2 with parameters µ 1, µ 2, Σ 1 and Σ 2 If Σ 1 Σ 2 we classify x π 1 if d1 Q(x) > d 2 Q (x) where d Q j (x) = 1 2 log Σ j 1 2 (x µ j) Σ 1 j (x µ j ) If Σ 1 = Σ 2 = Σ we classify x π 1 if d1 L(x) > d 2 L (x) where d L j (x) = µ j Σ 1 x 1 2 µ j Σ 1 µ j Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
20 Discriminant Analysis (Randles et al. 1978; Hawkins and McLachlan, 1997; Croux and Dehon, 2001; Hubert and Van Driessen, 2004) Populations π 1, π 2 with parameters µ 1, µ 2, Σ 1 and Σ 2 If Σ 1 Σ 2 we classify x π 1 if d1 Q(x) > d 2 Q (x) where d Q j (x) = 1 2 log Σ j 1 2 (x µ j) Σ 1 j (x µ j ) If Σ 1 = Σ 2 = Σ we classify x π 1 if d1 L(x) > d 2 L (x) where d L j (x) = µ j Σ 1 x 1 2 µ j Σ 1 µ j Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
21 Discriminant Analysis (Randles et al. 1978; Hawkins and McLachlan, 1997; Croux and Dehon, 2001; Hubert and Van Driessen, 2004) Populations π 1, π 2 with parameters µ 1, µ 2, Σ 1 and Σ 2 If Σ 1 Σ 2 we classify x π 1 if d1 Q(x) > d 2 Q (x) where d Q j (x) = 1 2 log Σ j 1 2 (x µ j) Σ 1 j (x µ j ) If Σ 1 = Σ 2 = Σ we classify x π 1 if d1 L(x) > d 2 L (x) where d L j (x) = µ j Σ 1 x 1 2 µ j Σ 1 µ j Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
22 Short intro to robust multivariate estimators Let X 1,..., X n R p f (x, µ, Σ) h (d(x, µ, Σ)) d(x, µ, Σ) = (x µ) Σ 1 (x µ) d i = d(x i, µ, Σ) = (x i µ) Σ 1 (x i µ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
23 Short intro to robust multivariate estimators The MLE estimators solve n w(d i ) (x i ˆµ) = 0 i=1 n w(d i ) (x i ˆµ) (x i ˆµ) = ˆΣ i=1 where W (d) = 2h (d)/h(d). Multivariate normal, W (d) = 1 Multivariate T η, W (d) = (p + η)/(d + η) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
24 Short intro to robust multivariate estimators The MLE estimators solve n w(d i ) (x i ˆµ) = 0 i=1 n w(d i ) (x i ˆµ) (x i ˆµ) = ˆΣ i=1 where W (d) = 2h (d)/h(d). Multivariate normal, W (d) = 1 Multivariate T η, W (d) = (p + η)/(d + η) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
25 Short intro to robust multivariate estimators The MLE estimators solve n w(d i ) (x i ˆµ) = 0 i=1 n w(d i ) (x i ˆµ) (x i ˆµ) = ˆΣ i=1 where W (d) = 2h (d)/h(d). Multivariate normal, W (d) = 1 Multivariate T η, W (d) = (p + η)/(d + η) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
26 M-estimators M-estimators (Maronna, 1976) n w 1 (d i ) (x i ˆµ) = 0 i=1 n w 2 (d i ) (x i ˆµ) (x i ˆµ) = ˆΣ i=1 S-estimators Davies (1987) subject to Σ = 1 min σ (d(x, µ, Σ)) µ,σ Stahel-Donoho Stahel (1981), Donoho (1982) computational complexity in p > 2 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
27 M-estimators M-estimators (Maronna, 1976) n w 1 (d i ) (x i ˆµ) = 0 i=1 n w 2 (d i ) (x i ˆµ) (x i ˆµ) = ˆΣ i=1 S-estimators Davies (1987) subject to Σ = 1 min σ (d(x, µ, Σ)) µ,σ Stahel-Donoho Stahel (1981), Donoho (1982) computational complexity in p > 2 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
28 M-estimators M-estimators (Maronna, 1976) n w 1 (d i ) (x i ˆµ) = 0 i=1 n w 2 (d i ) (x i ˆµ) (x i ˆµ) = ˆΣ i=1 S-estimators Davies (1987) subject to Σ = 1 min σ (d(x, µ, Σ)) µ,σ Stahel-Donoho Stahel (1981), Donoho (1982) computational complexity in p > 2 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
29 MVE (Minimum Volume Ellipsoid) (Rousseeuw) σ(d 1,..., d n ) = median(d 1,..., d n ) MCD (Minimum Covariance Determinant) (Rousseeuw) σ(d 1,..., d n ) = h i=1 d (i) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
30 MVE (Minimum Volume Ellipsoid) (Rousseeuw) σ(d 1,..., d n ) = median(d 1,..., d n ) MCD (Minimum Covariance Determinant) (Rousseeuw) σ(d 1,..., d n ) = h i=1 d (i) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
31 S-estimators (Davies, 1987) solve 1 n n ρ (d i /σ(d)) = b i=1 n w(d i /σ) (x i ˆµ) = 0 i=1 n w(d i /σ) (x i ˆµ) (x i ˆµ) = c ˆΣ i=1 with w(d) = ρ (d). Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
32 MM-estimators M-estimators with auxiliary scale (Tatsuoka and Tyler, 2000), also (Lopuhaä, 1992) Let ( µ n, Σ n ) be S-estimators and σ n = Σ subject to Γ = 1 min µ,γ n ρ 1 ((x i µ) Γ 1 (x i µ)/ σ n ) i=1 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
33 Back to Discriminant Analysis ˆd Q j or ˆd L j Plug-in strategy based on S-multivariate estimators ˆµ j and ˆΣ j ˆd Q j (x) = 1 2 log ˆΣ j 1 2 (x ˆµ j) ˆΣ 1 j (x ˆµ j ) ˆd L j (x) = ˆµ j Σ 1 x 1 2 ˆµ j Σ 1 ˆµ j j = 1, 2 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
34 Back to Discriminant Analysis ˆd Q j or ˆd L j Plug-in strategy based on S-multivariate estimators ˆµ j and ˆΣ j ˆd Q j (x) = 1 2 log ˆΣ j 1 2 (x ˆµ j) ˆΣ 1 j (x ˆµ j ) ˆd L j (x) = ˆµ j Σ 1 x 1 2 ˆµ j Σ 1 ˆµ j j = 1, 2 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
35 When Σ 1 = Σ 2 Pool ˆΣ = n 1 ˆΣ1 + n 2 ˆΣ2 /(n 1 + n 2 ) 2-sample S-estimators (He and Fung, 2002) 1 n 1 n 1 i=1 ( ρ d (1) i ) + 1 n 2 n 2 j=1 ( ρ d (2) j ) = b Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
36 When Σ 1 = Σ 2 Pool ˆΣ = n 1 ˆΣ1 + n 2 ˆΣ2 /(n 1 + n 2 ) 2-sample S-estimators (He and Fung, 2002) 1 n 1 n 1 i=1 ( ρ d (1) i ) + 1 n 2 n 2 j=1 ( ρ d (2) j ) = b Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
37 When Σ 1 = Σ 2 Pool ˆΣ = n 1 ˆΣ1 + n 2 ˆΣ2 /(n 1 + n 2 ) 2-sample S-estimators (He and Fung, 2002) 1 n 1 n 1 i=1 ( ρ d (1) i ) + 1 n 2 n 2 j=1 ( ρ d (2) j ) = b Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
38 Estimating misclassification error Resubstitution evaluate rule on the data (êr ) Cross-validation computing time Split into training and validation set (Hubert and Van Driessen, 2004) Bootstrap re-compute rule and evaluate on points outside the bootstrap sample (ê B ) (Efron, 1986) ê0.632 = ê B ê r Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
39 Estimating misclassification error Resubstitution evaluate rule on the data (êr ) Cross-validation computing time Split into training and validation set (Hubert and Van Driessen, 2004) Bootstrap re-compute rule and evaluate on points outside the bootstrap sample (ê B ) (Efron, 1986) ê0.632 = ê B ê r Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
40 Estimating misclassification error Resubstitution evaluate rule on the data (êr ) Cross-validation computing time Split into training and validation set (Hubert and Van Driessen, 2004) Bootstrap re-compute rule and evaluate on points outside the bootstrap sample (ê B ) (Efron, 1986) ê0.632 = ê B ê r Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
41 Estimating misclassification error Resubstitution evaluate rule on the data (êr ) Cross-validation computing time Split into training and validation set (Hubert and Van Driessen, 2004) Bootstrap re-compute rule and evaluate on points outside the bootstrap sample (ê B ) (Efron, 1986) ê0.632 = ê B ê r Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
42 Estimating misclassification error Resubstitution evaluate rule on the data (êr ) Cross-validation computing time Split into training and validation set (Hubert and Van Driessen, 2004) Bootstrap re-compute rule and evaluate on points outside the bootstrap sample (ê B ) (Efron, 1986) ê0.632 = ê B ê r Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
43 Estimating misclassification error Resubstitution evaluate rule on the data (êr ) Cross-validation computing time Split into training and validation set (Hubert and Van Driessen, 2004) Bootstrap re-compute rule and evaluate on points outside the bootstrap sample (ê B ) (Efron, 1986) ê0.632 = ê B ê r Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
44 Simulation Study p = 3 π 1 π 2 A: 50 N(0, I) 50 N(1, I) B: 40 N(0, I) + 10 N(5, I) 40 N(1, I) + 10 N( 4, I) C: 80 N(0, I) + 20 N(5, I) 8 N(1, I) + 2 N( 4, I) D: 16 N(0, I) + 4 N(0, 25 2 I) 16 N(1, I) + 4 N(1, 25 2 I) E: 58 N(0, I) + 12 N(5, I) 25 N(1, 4I) + 5 N( 10, I) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
45 Root mean squared error of the missclassification error A B C D E time B = error d B = (FRB) B = B = error d B = (Classical) B = k = error d CV k = k = n error d resub True error rate samples 50% BP S-multivariate estimators p = 3 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
46 Tests for nested linear hypotheses Scores-type tests (Markatou et al. 1991) y i = x i β + ɛ i, i = 1,..., n, β = (β 1, β 2) H 0 : β 2 = 0 versus H a : β 2 0 W 2 n = n 1 S n (ˆβ (0) n ) t Û 1 S n (ˆβ (0) n ) S n (ˆβ (0) n ) = n ρ 1((y i x i (1) i=1 ˆβ (0) n )/ˆσ n ) x i (2) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
47 Scores-type tests where Û = ˆQ 22 ˆM 21 ˆM 1 11 ˆQ 12 ˆQ 21 ˆM 1 11 ˆM 12 + ˆM 21 ˆM 1 11 ˆQ 11 ˆM 1 11 ˆM 12. and [ M = E[ρ 1(r) xx M11 M ] = 12 M 21 M 22 ], [ ] Q = E[ρ 1(r) 2 xx Q11 Q ] = 12, Q 21 Q 22 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
48 Example with asymmetric outliers Empirical distribution of the test statistic 1,000 random samples from y i = β 0 + β x i + ɛ i, i = 1,..., 100 β 0 = 1, β = (1, 1, 0, 0, 0, 0) F e (x) = 0.70 Φ (x) Φ ((x 4)/ 0.2) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
49 QQplot scores-type test statistic Chi-squared quantiles W2 quantiles Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
50 Bootstrapping test statistics Bootstrapping test statistics (Fisher and Hall, 1990; Hall and Wilson, 1991) Null data : ỹ i = x i ˆβ (0) n + r (a) i Bootstrap samples under H 0 : ỹi = x (0) i ˆβ n + r (a) i Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
51 R (0) Let β n and H a respectively. β R (a) n W 2 R n be the Robust Bootstrap re-calculations under H 0 and = n 1 S R n ( β S R n (β) = n i=1 ˆp = # R (0) n ρ 1((ỹ i R (0) ) [U R ] 1 S R n ( β n ), β x i ) / σ (a) n ) x i (2) { }/ Wn 2 R > Wn 2 B, Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
52 Example y = β 0 + β x + e, F e (u) = 0.8 Φ(u) Φ((u 5)/0.2) β j = 1, j = 0,..., 20, β j = 0, j = 21,..., 40 n = 5000 x N (0, I) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
53 Standardized residuals Residuals Index Outliers are well detected Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
54 Example H 0 : β j = 0 j = 21,..., 40 vs H a : β i 0 for some i = 21,..., 40 > library(robustbase) > m0 <- lmrob(y x0, x=true, y=true) > ma <- lmrob(y xa, x=true, y=true) > st <- scores.test(m0, ma) > st$a.p # Asymptotic chiˆ2 approximation [1] 0.02 > st$b.p [1] 0.44 # Robust Bootstrap approximation Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
55 Simulation results n p 0 p a ɛ ˆα A ˆα RB Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
56 Cleaning the data Clean the data and then perform MLE... as if nothing happened Objective versus subjective rules Subjective rules are intractable (Relles and Rogers, 1977, Monte Carlo!) Objective rules (Dupuis and Hamilton, 2000) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
57 Cleaning the data Clean the data and then perform MLE... as if nothing happened Objective versus subjective rules Subjective rules are intractable (Relles and Rogers, 1977, Monte Carlo!) Objective rules (Dupuis and Hamilton, 2000) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
58 Cleaning the data Clean the data and then perform MLE... as if nothing happened Objective versus subjective rules Subjective rules are intractable (Relles and Rogers, 1977, Monte Carlo!) Objective rules (Dupuis and Hamilton, 2000) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
59 Cleaning the data Clean the data and then perform MLE... as if nothing happened Objective versus subjective rules Subjective rules are intractable (Relles and Rogers, 1977, Monte Carlo!) Objective rules (Dupuis and Hamilton, 2000) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
60 Cleaning the data Clean the data and then perform MLE... as if nothing happened Objective versus subjective rules Subjective rules are intractable (Relles and Rogers, 1977, Monte Carlo!) Objective rules (Dupuis and Hamilton, 2000) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
61 Hard-rejection rules Fit a robust estimator Calculate a robust estimate of the standard deviation of the residuals, ˆσ Fix a number c > 0 and drop any observation with a residual larger than c ˆσ (typically 2 c 3). Apply classical methods to the remaining data Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
62 Hard-rejection rules Fit a robust estimator Calculate a robust estimate of the standard deviation of the residuals, ˆσ Fix a number c > 0 and drop any observation with a residual larger than c ˆσ (typically 2 c 3). Apply classical methods to the remaining data Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
63 Monte Carlo samples n β HRR estimates Monte Carlo estimate p = 2 20 β (0.046) β (0.051) β (0.017) β (0.017) p = 4 20 β (0.057) β (0.051) β (0.054) β (0.056) β (0.018) β (0.019) β (0.019) β (0.018) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
64 Robust confidence intervals Adrover, S-B, Zamar (2004) H ɛ = {F = (1 ɛ) F µ,σ + ɛ H} F µ,σ (u) = F 0 ((u µ) /σ) (L n, U n ) is globally robust if: 1 Stable lim n inf P F (L n < θ < U n ) (1 α) F H ɛ 2 Informative lim sup (U n L n ) < n F H ɛ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
65 Robust confidence intervals Adrover, S-B, Zamar (2004) H ɛ = {F = (1 ɛ) F µ,σ + ɛ H} F µ,σ (u) = F 0 ((u µ) /σ) (L n, U n ) is globally robust if: 1 Stable lim n inf P F (L n < θ < U n ) (1 α) F H ɛ 2 Informative lim sup (U n L n ) < n F H ɛ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
66 Robust confidence intervals Adrover, S-B, Zamar (2004) H ɛ = {F = (1 ɛ) F µ,σ + ɛ H} F µ,σ (u) = F 0 ((u µ) /σ) (L n, U n ) is globally robust if: 1 Stable lim n inf P F (L n < θ < U n ) (1 α) F H ɛ 2 Informative lim sup (U n L n ) < n F H ɛ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
67 Robust confidence intervals Adrover, S-B, Zamar (2004) H ɛ = {F = (1 ɛ) F µ,σ + ɛ H} F µ,σ (u) = F 0 ((u µ) /σ) (L n, U n ) is globally robust if: 1 Stable lim n inf P F (L n < θ < U n ) (1 α) F H ɛ 2 Informative lim sup (U n L n ) < n F H ɛ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
68 Simple examples that do not work X n ± t α/2 (n 1) S n/ n Sn 2 = 1 n ( Xi n 1 X ) 2 n i=1 Consider F x0 = (1 ɛ)f 0 + ɛδ x0. Then X n ± t α/2 (n 1) S n/ n a.s. ɛ x 0 > 0 n lim n inf P F (L n < 0 < U n ) lim P Fx0 (L n < 0 < U n ) = 0 F H ɛ n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
69 Simple examples that do not work X n ± t α/2 (n 1) S n/ n Sn 2 = 1 n ( Xi n 1 X ) 2 n i=1 Consider F x0 = (1 ɛ)f 0 + ɛδ x0. Then X n ± t α/2 (n 1) S n/ n a.s. ɛ x 0 > 0 n lim n inf P F (L n < 0 < U n ) lim P Fx0 (L n < 0 < U n ) = 0 F H ɛ n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
70 Now consider F x0 = (1 ɛ)f 0 + ɛ [δ x0 /2 + δ x0 /2] lim sup (U n L n ) n F H ɛ lim sup (U n L n ) = + n x 0 R + If we replace X n and S n by robust ˆµ n and ˆσ, ˆµ n ± 1.96ˆσ/ n informative not stable Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
71 Now consider F x0 = (1 ɛ)f 0 + ɛ [δ x0 /2 + δ x0 /2] lim sup (U n L n ) n F H ɛ lim sup (U n L n ) = + n x 0 R + If we replace X n and S n by robust ˆµ n and ˆσ, ˆµ n ± 1.96ˆσ/ n informative not stable Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
72 Now consider F x0 = (1 ɛ)f 0 + ɛ [δ x0 /2 + δ x0 /2] lim sup (U n L n ) n F H ɛ lim sup (U n L n ) = + n x 0 R + If we replace X n and S n by robust ˆµ n and ˆσ, ˆµ n ± 1.96ˆσ/ n informative not stable Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
73 Let r n and l n satisfy P F ( r n ˆµ n µ l n ) = 1 α (ˆµ n l n, ˆµ n + r n ) P F ( r n ˆµ n µ(f ) + µ(f ) µ l n ) = ( rn b P F ˆµ n µ(f ) l ) n b = v n v n v n b = µ(f ) µ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
74 Let r n and l n satisfy P F ( r n ˆµ n µ l n ) = 1 α (ˆµ n l n, ˆµ n + r n ) P F ( r n ˆµ n µ(f ) + µ(f ) µ l n ) = ( rn b P F ˆµ n µ(f ) l ) n b = v n v n v n b = µ(f ) µ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
75 Let r n and l n satisfy P F ( r n ˆµ n µ l n ) = 1 α (ˆµ n l n, ˆµ n + r n ) P F ( r n ˆµ n µ(f ) + µ(f ) µ l n ) = ( rn b P F ˆµ n µ(f ) l ) n b = v n v n v n b = µ(f ) µ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
76 Let r n and l n satisfy P F ( r n ˆµ n µ l n ) = 1 α (ˆµ n l n, ˆµ n + r n ) P F ( r n ˆµ n µ(f ) + µ(f ) µ l n ) = ( rn b P F ˆµ n µ(f ) l ) n b = v n v n v n b = µ(f ) µ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
77 Assume n (ˆµn µ(f )) D n N ( 0, V 2 (F ) ) F H ɛ µ(f ) µ < µ F H ɛ (if we knew b) ( ) ( ) ln b rn + b Φ + Φ = 1 α v n v n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
78 Assume n (ˆµn µ(f )) D n N ( 0, V 2 (F ) ) F H ɛ µ(f ) µ < µ F H ɛ (if we knew b) ( ) ( ) ln b rn + b Φ + Φ = 1 α v n v n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
79 Assume n (ˆµn µ(f )) D n N ( 0, V 2 (F ) ) F H ɛ µ(f ) µ < µ F H ɛ (if we knew b) ( ) ( ) ln b rn + b Φ + Φ = 1 α v n v n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
80 r n = v n z α/2 b l n = v n z α/2 + b (ˆµn v n z α/2 b, ˆµ n + v n z α/2 b ) (ˆµn v n z α/2 µ, ˆµ n + v n z α/2 + µ ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
81 r n = v n z α/2 b l n = v n z α/2 + b (ˆµn v n z α/2 b, ˆµ n + v n z α/2 b ) (ˆµn v n z α/2 µ, ˆµ n + v n z α/2 + µ ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
82 Alternative Fraiman et al. (2001) P F ( ˆµ n µ q n ) = 1 α ( ) ˆqn b Φ + Φ v n ( ˆqn + b v n ) 1 = 1 α q n (b 1 ) < q n (b 2 ) if b 1 < b 2 ( ) qn µ Φ + Φ v n ( qn + µ v n ) 1 = 1 α These intervals are shorter: ˆµ n ± q n v n z α/2 + µ > q n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
83 p-values H 0 : µ µ 0 versus H 1 : µ > µ 0 Reject H 0 if ˆµ > µ 0 + v n Φ 1 (1 α) b(f ) sup P F (ˆµ > µ0 + v n Φ 1 (1 α) b(f ) ) = µ µ 0 sup P F (ˆµ µ > µ0 µ + v n Φ 1 (1 α) b(f ) ) µ µ 0 P F (ˆµ µ0 > µ 0 µ 0 + v n Φ 1 (1 α) b(f ) ) = α Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
84 p-values H 0 : µ µ 0 versus H 1 : µ > µ 0 Reject H 0 if ˆµ > µ 0 + v n Φ 1 (1 α) b(f ) sup P F (ˆµ > µ0 + v n Φ 1 (1 α) b(f ) ) = µ µ 0 sup P F (ˆµ µ > µ0 µ + v n Φ 1 (1 α) b(f ) ) µ µ 0 P F (ˆµ µ0 > µ 0 µ 0 + v n Φ 1 (1 α) b(f ) ) = α Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
85 p-values H 0 : µ µ 0 versus H 1 : µ > µ 0 Reject H 0 if ˆµ > µ 0 + v n Φ 1 (1 α) b(f ) sup P F (ˆµ > µ0 + v n Φ 1 (1 α) b(f ) ) = µ µ 0 sup P F (ˆµ µ > µ0 µ + v n Φ 1 (1 α) b(f ) ) µ µ 0 P F (ˆµ µ0 > µ 0 µ 0 + v n Φ 1 (1 α) b(f ) ) = α Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
86 { } p-value = inf α : ˆµ > µ 0 + v n Φ 1 (1 α) b(f ) ( ) ˆµ µ0 b(f ) 1 Φ v n ( ) ˆµ µ0 µ 1 Φ v n Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
87 µ is not the maximum (asymptotic) bias B F0 (ɛ) = sup µ(f ) µ(f 0 ) /σ 0 F H ɛ(f 0 ) µ = k ˆσ B F0 (ɛ) k = σ (F 0 ) sup F H ɛ(f 0 ) σ (F ) = 1 σ 1 (ɛ) σ 1 (ɛ) = σ (F ) inf F H ɛ(f 0 ) σ (F 0 ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
88 µ is not the maximum (asymptotic) bias B F0 (ɛ) = sup µ(f ) µ(f 0 ) /σ 0 F H ɛ(f 0 ) µ = k ˆσ B F0 (ɛ) k = σ (F 0 ) sup F H ɛ(f 0 ) σ (F ) = 1 σ 1 (ɛ) σ 1 (ɛ) = σ (F ) inf F H ɛ(f 0 ) σ (F 0 ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
89 µ is not the maximum (asymptotic) bias B F0 (ɛ) = sup µ(f ) µ(f 0 ) /σ 0 F H ɛ(f 0 ) µ = k ˆσ B F0 (ɛ) k = σ (F 0 ) sup F H ɛ(f 0 ) σ (F ) = 1 σ 1 (ɛ) σ 1 (ɛ) = σ (F ) inf F H ɛ(f 0 ) σ (F 0 ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
90 µ is not the maximum (asymptotic) bias B F0 (ɛ) = sup µ(f ) µ(f 0 ) /σ 0 F H ɛ(f 0 ) µ = k ˆσ B F0 (ɛ) k = σ (F 0 ) sup F H ɛ(f 0 ) σ (F ) = 1 σ 1 (ɛ) σ 1 (ɛ) = σ (F ) inf F H ɛ(f 0 ) σ (F 0 ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
91 Finite sample behaviour Location - scale model n i=1 Ψ D ( ) Yi ˆµ = 0 ˆσ Ψ D c length-optimal (Fraiman et al. 2001) ˆσ is an S-scale estimator Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
92 ɛ n Globally Robust Naive x 0 = 1.5 x 0 = 4.0 x 0 = 1.5 x 0 = (0.88) 0.94 (0.88) 0.94 (0.88) 0.94 (0.88) (0.64) 0.94 (0.64) 0.94 (0.64) 0.94 (0.64) (0.41) 0.94 (0.41) 0.94 (0.41) 0.94 (0.41) (0.18) 0.93 (0.18) 0.93 (0.18) 0.93 (0.18) (0.89) 0.94 (0.88) 0.94 (0.89) 0.94 (0.89) (0.64) 0.95 (0.64) 0.95 (0.64) 0.95 (0.63) (0.41) 0.95 (0.41) 0.95 (0.41) 0.95 (0.41) (0.19) 0.95 (0.19) 0.94 (0.18) 0.94 (0.18) (0.95) 0.93 (0.94) 0.93 (0.91) 0.93 (0.91) (0.74) 0.95 (0.73) 0.93 (0.67) 0.93 (0.67) (0.54) 0.95 (0.54) 0.88 (0.43) 0.88 (0.44) (0.35) 0.96 (0.35) 0.62 (0.20) 0.61 (0.20) (1.15) 0.95 (1.21) 0.91 (0.96) 0.92 (1.01) (0.98) 0.97 (1.03) 0.86 (0.70) 0.88 (0.75) (0.81) 0.98 (0.86) 0.65 (0.44) 0.69 (0.49) (0.63) 1.00 (0.66) 0.05 (0.20) 0.05 (0.22) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
93 Simple linear regression Asymptotic Distribution n (ˆβ n β(f )) D n N (0, V (F )) Bias Bound sup ˆβ n β(f ) = σ e B 1 (ɛ) σ x F H ɛ Computable and small maximum bias Normal distribution over the whole contamination neighbourhood Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
94 Generalized Median of Slopes (Brown and Mood, 1951; Adrover, S-B, Zamar, 2004) n ( ) sign y i ˆα ˆβ (x i ˆµ x ) sign (x i ˆµ x ) = 0 i=1 n ( ) sign y i ˆα ˆβ (x i ˆµ x ) i=1 = 0 where ˆµ x = median (x 1,..., x n ) Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
95 ɛ MS GMS B 1 (ɛ) MS = median (y i /x i ) Minimax (asymptotic) bias for Y = β X + ɛ Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
96 ɛ n Mild Medium Strong (2.01) 0.99 (2.01) 0.99 (2.01) (1.19) 0.97 (1.19) 0.97 (1.27) 5% (0.97) 0.96 (0.98) 0.97 (1.03) (0.85) 0.94 (0.85) 0.97 (0.89) (0.76) 0.95 (0.77) 0.97 (0.81) (0.61) 0.96 (0.61) 0.97 (0.62) (2.00) 0.99 (2.02) 0.99 (2.43) (1.34) 0.98 (1.36) 0.99 (1.69) 10% (1.17) 0.94 (1.19) 0.99 (1.44) (1.08) 0.92 (1.11) 0.99 (1.30) (1.01) 0.92 (1.05) 0.99 (1.20) (0.87) 0.93 (0.92) 0.99 (1.00) Bootstrap standard deviations Mild: (3, 1.5[2]) Medium: (5, 2.5) Strong: (5, 15) β = 0 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
97 Example Motorola returns Motorola shares versus 30-day US Treasury bills January 1978 to December 1987 Model Motorola i = α + β Market i + ɛ i Point estimates Estimator ˆβ GMS 1.25 MM 1.34 LS 0.85 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
98 Market Return Motorola Return MM GMS LS Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
99 Hypothesis of interest H 0 : β > 1 versus H a : β 1 ɛ 1/ ˆσ e /ˆσ x = 1.41 Bias bound: ˆ β = p-value 1 Φ (( ) /0.169) = Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
100 ɛ n Mild Medium Strong (1.28) 0.90 (1.30) 0.89 (1.33) (0.98) 0.92 (1.00) 0.92 (1.02) 5% (0.84) 0.91 (0.85) 0.93 (0.87) (0.75) 0.92 (0.77) 0.94 (0.78) (0.69) 0.92 (0.71) 0.94 (0.72) (055) 0.94 (0.56) 0.95 (0.57) (1.42) 0.91 (1.46) 0.92 (1.64) (1.13) 0.91 (1.18) 0.94 (1.33) 10% (1.01) 0.90 (1.09) 0.95 (1.20) (0.95) 0.89 (1.01) 0.96 (1.10) (0.89) 0.88 (0.96) 0.97 (1.05) (0.76) 0.90 (0.84) 0.98 (0.90) Empirical approximation to the asymptotic variance Mild: (3, 1.5[2]) Medium: (5, 2.5) Strong: (5, 15) β = 0 Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
101 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
102 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
103 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
104 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
105 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
106 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
107 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
108 Difficulty of estimating asymptotic variance Need asymptotic normal distribution for all F H ɛ Need bias bounds error scale estimation correction for scale estimation yields even longer CIs Can we avoid using bias bounds? Can we avoid requiring global asymptotic distribution? Can we avoid estimating the SD of the estimator? Matias Salibian-Barrera (UBC) Robust inference (2) ECARES - Dec / 65
Globally Robust Inference
Globally Robust Inference Jorge Adrover Univ. Nacional de Cordoba and CIEM, Argentina Jose Ramon Berrendero Universidad Autonoma de Madrid, Spain Matias Salibian-Barrera Carleton University, Canada Ruben
Journal of Statistical Software
JSS Journal of Statistical Software April 2013, Volume 53, Issue 3. http://www.jstatsoft.org/ Fast and Robust Bootstrap for Multivariate Inference: The R Package FRB Stefan Van Aelst Ghent University Gert
Fast and robust bootstrap for LTS
Fast and robust bootstrap for LTS Gert Willems a,, Stefan Van Aelst b a Department of Mathematics and Computer Science, University of Antwerp, Middelheimlaan 1, B-2020 Antwerp, Belgium b Department of
MULTIVARIATE TECHNIQUES, ROBUSTNESS
MULTIVARIATE TECHNIQUES, ROBUSTNESS Mia Hubert Associate Professor, Department of Mathematics and L-STAT Katholieke Universiteit Leuven, Belgium mia.hubert@wis.kuleuven.be Peter J. Rousseeuw 1 Senior Researcher,
Small Sample Corrections for LTS and MCD
myjournal manuscript No. (will be inserted by the editor) Small Sample Corrections for LTS and MCD G. Pison, S. Van Aelst, and G. Willems Department of Mathematics and Computer Science, Universitaire Instelling
Accurate and Powerful Multivariate Outlier Detection
Int. Statistical Inst.: Proc. 58th World Statistical Congress, 11, Dublin (Session CPS66) p.568 Accurate and Powerful Multivariate Outlier Detection Cerioli, Andrea Università di Parma, Dipartimento di
Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
IMPROVING THE SMALL-SAMPLE EFFICIENCY OF A ROBUST CORRELATION MATRIX: A NOTE
IMPROVING THE SMALL-SAMPLE EFFICIENCY OF A ROBUST CORRELATION MATRIX: A NOTE Eric Blankmeyer Department of Finance and Economics McCoy College of Business Administration Texas State University San Marcos
Development of robust scatter estimators under independent contamination model
Development of robust scatter estimators under independent contamination model C. Agostinelli 1, A. Leung, V.J. Yohai 3 and R.H. Zamar 1 Universita Cà Foscàri di Venezia, University of British Columbia,
Fast and Robust Discriminant Analysis
Fast and Robust Discriminant Analysis Mia Hubert a,1, Katrien Van Driessen b a Department of Mathematics, Katholieke Universiteit Leuven, W. De Croylaan 54, B-3001 Leuven. b UFSIA-RUCA Faculty of Applied
9. Robust regression
9. Robust regression Least squares regression........................................................ 2 Problems with LS regression..................................................... 3 Robust regression............................................................
Chapter 7. Hypothesis Testing
Chapter 7. Hypothesis Testing Joonpyo Kim June 24, 2017 Joonpyo Kim Ch7 June 24, 2017 1 / 63 Basic Concepts of Testing Suppose that our interest centers on a random variable X which has density function
Homework 2: Simple Linear Regression
STAT 4385 Applied Regression Analysis Homework : Simple Linear Regression (Simple Linear Regression) Thirty (n = 30) College graduates who have recently entered the job market. For each student, the CGPA
Predictive Regression and Robust Hypothesis Testing: Predictability Hidden by Anomalous Observations
Predictive Regression and Robust Hypothesis Testing: Predictability Hidden by Anomalous Observations Fabio Trojani University of Lugano and Swiss Finance Institute fabio.trojani@usi.ch Joint work with
CLASSIFICATION EFFICIENCIES FOR ROBUST LINEAR DISCRIMINANT ANALYSIS
Statistica Sinica 8(008), 58-599 CLASSIFICATION EFFICIENCIES FOR ROBUST LINEAR DISCRIMINANT ANALYSIS Christophe Croux, Peter Filzmoser and Kristel Joossens K.U. Leuven and Vienna University of Technology
UNIVERSITÄT POTSDAM Institut für Mathematik
UNIVERSITÄT POTSDAM Institut für Mathematik Testing the Acceleration Function in Life Time Models Hannelore Liero Matthias Liero Mathematische Statistik und Wahrscheinlichkeitstheorie Universität Potsdam
Simple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
Robust Estimation of Cronbach s Alpha
Robust Estimation of Cronbach s Alpha A. Christmann University of Dortmund, Fachbereich Statistik, 44421 Dortmund, Germany. S. Van Aelst Ghent University (UGENT), Department of Applied Mathematics and
Robust Inference for Seemingly Unrelated Regression Models
Robust Inference for Seemingly Unrelated Regression Models arxiv:1801.04716v3 [stat.me] 14 May 2018 Kris Peremans and Stefan Van Aelst Department of Mathematics, KU Leuven, 3001 Leuven, Belgium May 15,
On consistency factors and efficiency of robust S-estimators
TEST DOI.7/s749-4-357-7 ORIGINAL PAPER On consistency factors and efficiency of robust S-estimators Marco Riani Andrea Cerioli Francesca Torti Received: 5 May 3 / Accepted: 6 January 4 Sociedad de Estadística
odhady a jejich (ekonometrické)
modifikace Stochastic Modelling in Economics and Finance 2 Advisor : Prof. RNDr. Jan Ámos Víšek, CSc. Petr Jonáš 27 th April 2009 Contents 1 2 3 4 29 1 In classical approach we deal with stringent stochastic
Ch 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
Robust Wilks' Statistic based on RMCD for One-Way Multivariate Analysis of Variance (MANOVA)
ISSN 2224-584 (Paper) ISSN 2225-522 (Online) Vol.7, No.2, 27 Robust Wils' Statistic based on RMCD for One-Way Multivariate Analysis of Variance (MANOVA) Abdullah A. Ameen and Osama H. Abbas Department
Central Limit Theorem ( 5.3)
Central Limit Theorem ( 5.3) Let X 1, X 2,... be a sequence of independent random variables, each having n mean µ and variance σ 2. Then the distribution of the partial sum S n = X i i=1 becomes approximately
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)
ROBUST ESTIMATION OF A CORRELATION COEFFICIENT: AN ATTEMPT OF SURVEY
ROBUST ESTIMATION OF A CORRELATION COEFFICIENT: AN ATTEMPT OF SURVEY G.L. Shevlyakov, P.O. Smirnov St. Petersburg State Polytechnic University St.Petersburg, RUSSIA E-mail: Georgy.Shevlyakov@gmail.com
Applied Regression. Applied Regression. Chapter 2 Simple Linear Regression. Hongcheng Li. April, 6, 2013
Applied Regression Chapter 2 Simple Linear Regression Hongcheng Li April, 6, 2013 Outline 1 Introduction of simple linear regression 2 Scatter plot 3 Simple linear regression model 4 Test of Hypothesis
Bootstrap, Jackknife and other resampling methods
Bootstrap, Jackknife and other resampling methods Part VI: Cross-validation Rozenn Dahyot Room 128, Department of Statistics Trinity College Dublin, Ireland dahyot@mee.tcd.ie 2005 R. Dahyot (TCD) 453 Modern
Cross-Validation with Confidence
Cross-Validation with Confidence Jing Lei Department of Statistics, Carnegie Mellon University WHOA-PSI Workshop, St Louis, 2017 Quotes from Day 1 and Day 2 Good model or pure model? Occam s razor We really
Robust Linear Discriminant Analysis and the Projection Pursuit Approach
Robust Linear Discriminant Analysis and the Projection Pursuit Approach Practical aspects A. M. Pires 1 Department of Mathematics and Applied Mathematics Centre, Technical University of Lisbon (IST), Lisboa,
Computational Statistics and Data Analysis
Computational Statistics and Data Analysis 65 (2013) 29 45 Contents lists available at SciVerse ScienceDirect Computational Statistics and Data Analysis journal homepage: www.elsevier.com/locate/csda Robust
MS&E 226: Small Data
MS&E 226: Small Data Lecture 12: Frequentist properties of estimators (v4) Ramesh Johari ramesh.johari@stanford.edu 1 / 39 Frequentist inference 2 / 39 Thinking like a frequentist Suppose that for some
Cross-Validation with Confidence
Cross-Validation with Confidence Jing Lei Department of Statistics, Carnegie Mellon University UMN Statistics Seminar, Mar 30, 2017 Overview Parameter est. Model selection Point est. MLE, M-est.,... Cross-validation
MS-E2112 Multivariate Statistical Analysis (5cr) Lecture 4: Measures of Robustness, Robust Principal Component Analysis
MS-E2112 Multivariate Statistical Analysis (5cr) Lecture 4:, Robust Principal Component Analysis Contents Empirical Robust Statistical Methods In statistics, robust methods are methods that perform well
Graduate Econometrics I: Maximum Likelihood I
Graduate Econometrics I: Maximum Likelihood I Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood
Problem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
Simple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
The Adaptive Lasso and Its Oracle Properties Hui Zou (2006), JASA
The Adaptive Lasso and Its Oracle Properties Hui Zou (2006), JASA Presented by Dongjun Chung March 12, 2010 Introduction Definition Oracle Properties Computations Relationship: Nonnegative Garrote Extensions:
Introduction to Robust Statistics. Elvezio Ronchetti. Department of Econometrics University of Geneva Switzerland.
Introduction to Robust Statistics Elvezio Ronchetti Department of Econometrics University of Geneva Switzerland Elvezio.Ronchetti@metri.unige.ch http://www.unige.ch/ses/metri/ronchetti/ 1 Outline Introduction
Inference After Variable Selection
Department of Mathematics, SIU Carbondale Inference After Variable Selection Lasanthi Pelawa Watagoda lasanthi@siu.edu June 12, 2017 Outline 1 Introduction 2 Inference For Ridge and Lasso 3 Variable Selection
The S-estimator of multivariate location and scatter in Stata
The Stata Journal (yyyy) vv, Number ii, pp. 1 9 The S-estimator of multivariate location and scatter in Stata Vincenzo Verardi University of Namur (FUNDP) Center for Research in the Economics of Development
Robust estimation of scale and covariance with P n and its application to precision matrix estimation
Robust estimation of scale and covariance with P n and its application to precision matrix estimation Garth Tarr, Samuel Müller and Neville Weber USYD 2013 School of Mathematics and Statistics THE UNIVERSITY
BTRY 4090: Spring 2009 Theory of Statistics
BTRY 4090: Spring 2009 Theory of Statistics Guozhang Wang September 25, 2010 1 Review of Probability We begin with a real example of using probability to solve computationally intensive (or infeasible)
Inference for Regression
Inference for Regression Section 9.4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Department of Mathematics University of Houston Lecture 13b - 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu
Economics 582 Random Effects Estimation
Economics 582 Random Effects Estimation Eric Zivot May 29, 2013 Random Effects Model Hence, the model can be re-written as = x 0 β + + [x ] = 0 (no endogeneity) [ x ] = = + x 0 β + + [x ] = 0 [ x ] = 0
36. Multisample U-statistics and jointly distributed U-statistics Lehmann 6.1
36. Multisample U-statistics jointly distributed U-statistics Lehmann 6.1 In this topic, we generalize the idea of U-statistics in two different directions. First, we consider single U-statistics for situations
Statistics 135 Fall 2008 Final Exam
Name: SID: Statistics 135 Fall 2008 Final Exam Show your work. The number of points each question is worth is shown at the beginning of the question. There are 10 problems. 1. [2] The normal equations
Robustní monitorování stability v modelu CAPM
Robustní monitorování stability v modelu CAPM Ondřej Chochola, Marie Hušková, Zuzana Prášková (MFF UK) Josef Steinebach (University of Cologne) ROBUST 2012, Němčičky, 10.-14.9. 2012 Contents Introduction
STAT440/840: Statistical Computing
First Prev Next Last STAT440/840: Statistical Computing Paul Marriott pmarriott@math.uwaterloo.ca MC 6096 February 2, 2005 Page 1 of 41 First Prev Next Last Page 2 of 41 Chapter 3: Data resampling: the
Master s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 016 Full points may be obtained for correct answers to eight questions. Each numbered question which may have several parts is worth
Ch 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
UNIVERSITY OF MASSACHUSETTS. Department of Mathematics and Statistics. Basic Exam - Applied Statistics. Tuesday, January 17, 2017
UNIVERSITY OF MASSACHUSETTS Department of Mathematics and Statistics Basic Exam - Applied Statistics Tuesday, January 17, 2017 Work all problems 60 points are needed to pass at the Masters Level and 75
Minimum Hellinger Distance Estimation in a. Semiparametric Mixture Model
Minimum Hellinger Distance Estimation in a Semiparametric Mixture Model Sijia Xiang 1, Weixin Yao 1, and Jingjing Wu 2 1 Department of Statistics, Kansas State University, Manhattan, Kansas, USA 66506-0802.
One-Sample Numerical Data
One-Sample Numerical Data quantiles, boxplot, histogram, bootstrap confidence intervals, goodness-of-fit tests University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html
Introduction Robust regression Examples Conclusion. Robust regression. Jiří Franc
Robust regression Robust estimation of regression coefficients in linear regression model Jiří Franc Czech Technical University Faculty of Nuclear Sciences and Physical Engineering Department of Mathematics
Scatter plot of data from the study. Linear Regression
1 2 Linear Regression Scatter plot of data from the study. Consider a study to relate birthweight to the estriol level of pregnant women. The data is below. i Weight (g / 100) i Weight (g / 100) 1 7 25
STAT5044: Regression and Anova
STAT5044: Regression and Anova Inyoung Kim 1 / 49 Outline 1 How to check assumptions 2 / 49 Assumption Linearity: scatter plot, residual plot Randomness: Run test, Durbin-Watson test when the data can
Robust model selection criteria for robust S and LT S estimators
Hacettepe Journal of Mathematics and Statistics Volume 45 (1) (2016), 153 164 Robust model selection criteria for robust S and LT S estimators Meral Çetin Abstract Outliers and multi-collinearity often
K. Model Diagnostics. residuals ˆɛ ij = Y ij ˆµ i N = Y ij Ȳ i semi-studentized residuals ω ij = ˆɛ ij. studentized deleted residuals ɛ ij =
K. Model Diagnostics We ve already seen how to check model assumptions prior to fitting a one-way ANOVA. Diagnostics carried out after model fitting by using residuals are more informative for assessing
Can we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
2014/2015 Smester II ST5224 Final Exam Solution
014/015 Smester II ST54 Final Exam Solution 1 Suppose that (X 1,, X n ) is a random sample from a distribution with probability density function f(x; θ) = e (x θ) I [θ, ) (x) (i) Show that the family of
Confidence Intervals, Testing and ANOVA Summary
Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0
401 Review. 6. Power analysis for one/two-sample hypothesis tests and for correlation analysis.
401 Review Major topics of the course 1. Univariate analysis 2. Bivariate analysis 3. Simple linear regression 4. Linear algebra 5. Multiple regression analysis Major analysis methods 1. Graphical analysis
EXAMINERS REPORT & SOLUTIONS STATISTICS 1 (MATH 11400) May-June 2009
EAMINERS REPORT & SOLUTIONS STATISTICS (MATH 400) May-June 2009 Examiners Report A. Most plots were well done. Some candidates muddled hinges and quartiles and gave the wrong one. Generally candidates
LECTURE 5 HYPOTHESIS TESTING
October 25, 2016 LECTURE 5 HYPOTHESIS TESTING Basic concepts In this lecture we continue to discuss the normal classical linear regression defined by Assumptions A1-A5. Let θ Θ R d be a parameter of interest.
Non-Parametric Bootstrap Mean. Squared Error Estimation For M- Quantile Estimators Of Small Area. Averages, Quantiles And Poverty
Working Paper M11/02 Methodology Non-Parametric Bootstrap Mean Squared Error Estimation For M- Quantile Estimators Of Small Area Averages, Quantiles And Poverty Indicators Stefano Marchetti, Nikos Tzavidis,
Lecture 5: LDA and Logistic Regression
Lecture 5: and Logistic Regression Hao Helen Zhang Hao Helen Zhang Lecture 5: and Logistic Regression 1 / 39 Outline Linear Classification Methods Two Popular Linear Models for Classification Linear Discriminant
A Resampling Method on Pivotal Estimating Functions
A Resampling Method on Pivotal Estimating Functions Kun Nie Biostat 277,Winter 2004 March 17, 2004 Outline Introduction A General Resampling Method Examples - Quantile Regression -Rank Regression -Simulation
Why the Rousseeuw Yohai Paradigm is One of the Largest and Longest Running Scientific Hoaxes in History
Why the Rousseeuw Yohai Paradigm is One of the Largest and Longest Running Scientific Hoaxes in History David J. Olive Southern Illinois University December 4, 2012 Abstract The Rousseeuw Yohai paradigm
Chapter 3: Maximum Likelihood Theory
Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood
Simple Regression Model Setup Estimation Inference Prediction. Model Diagnostic. Multiple Regression. Model Setup and Estimation.
Statistical Computation Math 475 Jimin Ding Department of Mathematics Washington University in St. Louis www.math.wustl.edu/ jmding/math475/index.html October 10, 2013 Ridge Part IV October 10, 2013 1
A Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
Scatter plot of data from the study. Linear Regression
1 2 Linear Regression Scatter plot of data from the study. Consider a study to relate birthweight to the estriol level of pregnant women. The data is below. i Weight (g / 100) i Weight (g / 100) 1 7 25
Lecture 12 Robust Estimation
Lecture 12 Robust Estimation Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of Karlsruhe Financial Econometrics, Summer Semester 2007 Copyright These lecture-notes
Robust Standard Errors for Robust Estimators
Robust Standard Errors for Robust Estimators Christophe Croux K.U.Leuven Geert Dhaene K.U.Leuven Dirk Hoorelbeke K.U.Leuven Abstract A regression estimator is said to be robust if it is still reliable
Applied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
Math 5305 Notes. Diagnostics and Remedial Measures. Jesse Crawford. Department of Mathematics Tarleton State University
Math 5305 Notes Diagnostics and Remedial Measures Jesse Crawford Department of Mathematics Tarleton State University (Tarleton State University) Diagnostics and Remedial Measures 1 / 44 Model Assumptions
Introductory Econometrics
Based on the textbook by Wooldridge: : A Modern Approach Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna November 23, 2013 Outline Introduction
Review. December 4 th, Review
December 4 th, 2017 Att. Final exam: Course evaluation Friday, 12/14/2018, 10:30am 12:30pm Gore Hall 115 Overview Week 2 Week 4 Week 7 Week 10 Week 12 Chapter 6: Statistics and Sampling Distributions Chapter
The minimum volume ellipsoid (MVE), introduced
Minimum volume ellipsoid Stefan Van Aelst 1 and Peter Rousseeuw 2 The minimum volume ellipsoid (MVE) estimator is based on the smallest volume ellipsoid that covers h of the n observations. It is an affine
Statistical Inference
Statistical Inference Classical and Bayesian Methods Revision Class for Midterm Exam AMS-UCSC Th Feb 9, 2012 Winter 2012. Session 1 (Revision Class) AMS-132/206 Th Feb 9, 2012 1 / 23 Topics Topics We will
Supplement to Quantile-Based Nonparametric Inference for First-Price Auctions
Supplement to Quantile-Based Nonparametric Inference for First-Price Auctions Vadim Marmer University of British Columbia Artyom Shneyerov CIRANO, CIREQ, and Concordia University August 30, 2010 Abstract
Advanced Statistics II: Non Parametric Tests
Advanced Statistics II: Non Parametric Tests Aurélien Garivier ParisTech February 27, 2011 Outline Fitting a distribution Rank Tests for the comparison of two samples Two unrelated samples: Mann-Whitney
parameter space Θ, depending only on X, such that Note: it is not θ that is random, but the set C(X).
4. Interval estimation The goal for interval estimation is to specify the accurary of an estimate. A 1 α confidence set for a parameter θ is a set C(X) in the parameter space Θ, depending only on X, such
TITLE : Robust Control Charts for Monitoring Process Mean of. Phase-I Multivariate Individual Observations AUTHORS : Asokan Mulayath Variyath.
TITLE : Robust Control Charts for Monitoring Process Mean of Phase-I Multivariate Individual Observations AUTHORS : Asokan Mulayath Variyath Department of Mathematics and Statistics, Memorial University
Median Cross-Validation
Median Cross-Validation Chi-Wai Yu 1, and Bertrand Clarke 2 1 Department of Mathematics Hong Kong University of Science and Technology 2 Department of Medicine University of Miami IISA 2011 Outline Motivational
Robust Resampling Methods for Time Series
Robust Resampling Methods for Time Series Lorenzo Camponovo University of Lugano Olivier Scaillet University of Geneva and Swiss Finance Institute Fabio Trojani University of Lugano and Swiss Finance Institute
Statistics 203: Introduction to Regression and Analysis of Variance Course review
Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying
Small sample corrections for LTS and MCD
Metrika (2002) 55: 111 123 > Springer-Verlag 2002 Small sample corrections for LTS and MCD G. Pison, S. Van Aelst*, and G. Willems Department of Mathematics and Computer Science, Universitaire Instelling
The Glejser Test and the Median Regression
Sankhyā : The Indian Journal of Statistics Special Issue on Quantile Regression and Related Methods 2005, Volume 67, Part 2, pp 335-358 c 2005, Indian Statistical Institute The Glejser Test and the Median
1 Statistical inference for a population mean
1 Statistical inference for a population mean 1. Inference for a large sample, known variance Suppose X 1,..., X n represents a large random sample of data from a population with unknown mean µ and known
Data Integration for Big Data Analysis for finite population inference
for Big Data Analysis for finite population inference Jae-kwang Kim ISU January 23, 2018 1 / 36 What is big data? 2 / 36 Data do not speak for themselves Knowledge Reproducibility Information Intepretation
Stat 710: Mathematical Statistics Lecture 31
Stat 710: Mathematical Statistics Lecture 31 Jun Shao Department of Statistics University of Wisconsin Madison, WI 53706, USA Jun Shao (UW-Madison) Stat 710, Lecture 31 April 13, 2009 1 / 13 Lecture 31:
FAST CROSS-VALIDATION IN ROBUST PCA
COMPSTAT 2004 Symposium c Physica-Verlag/Springer 2004 FAST CROSS-VALIDATION IN ROBUST PCA Sanne Engelen, Mia Hubert Key words: Cross-Validation, Robustness, fast algorithm COMPSTAT 2004 section: Partial
Non-parametric bootstrap mean squared error estimation for M-quantile estimates of small area means, quantiles and poverty indicators
Non-parametric bootstrap mean squared error estimation for M-quantile estimates of small area means, quantiles and poverty indicators Stefano Marchetti 1 Nikos Tzavidis 2 Monica Pratesi 3 1,3 Department
F & B Approaches to a simple model
A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 215 http://www.astro.cornell.edu/~cordes/a6523 Lecture 11 Applications: Model comparison Challenges in large-scale surveys
Re-weighted Robust Control Charts for Individual Observations
Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 426 Re-weighted Robust Control Charts for Individual Observations Mandana Mohammadi 1, Habshah Midi 1,2 and Jayanthi Arasan 1,2 1 Laboratory of Applied
6-1. Canonical Correlation Analysis
6-1. Canonical Correlation Analysis Canonical Correlatin analysis focuses on the correlation between a linear combination of the variable in one set and a linear combination of the variables in another
Lecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2
Lecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2 Fall, 2013 Page 1 Random Variable and Probability Distribution Discrete random variable Y : Finite possible values {y
A measurement error model approach to small area estimation
A measurement error model approach to small area estimation Jae-kwang Kim 1 Spring, 2015 1 Joint work with Seunghwan Park and Seoyoung Kim Ouline Introduction Basic Theory Application to Korean LFS Discussion