Recitation 5. Inference and Power Calculations. Yiqing Xu. March 7, 2014 MIT

Transcription

1 Recitation 5 Inference and Power Calculations Yiqing Xu MIT March 7, 2014

2 1 Inference of Frequentists 2 Power Calculations

3 Inference (mostly MHE Ch8) Inference in Asymptopia (and with Weak Null) Robust standard error and biases Clustering Bootstrap Permutation inference (and with Sharp Null) Yiqing Xu (MIT) Rec5 March 7, / 16

4 Robust Standard Errors We re interested in the large-sample (approximate) distributions of the estimators Robust standard errors are robust because they provide accurate hypothesis tests and confidence intervals in large samples [ Xi X N( ˆβ ] 1 β) = i ( 1 Xi e i ) N N A.VarCov( ˆβ) = E[X i X i ] 1 E[X i X i e 2 i ] 1 E[X i X i ] 1 Conventional: E[X i X i e2 i ] = E[X ix i ] 1 σ 2 H. White: No! E[X i X i e2 i ] is pretty good, why bother? But they re biased (in finite samples)! Yiqing Xu (MIT) Rec5 March 7, / 16

5 Yiqing Xu (MIT) Rec5 March 7, / 16

6 Robust S.E.s are Biased! Robust S.E. has more (downward) bias in finte samples where heteroskedasticity is small than the conventional one Never to accept the result when robust S.E. is smaller than conventional Always be more conservative take the largest S.E. Yiqing Xu (MIT) Rec5 March 7, / 16

7 Clustering Moulton factor y ig = β 0 + β 1 x g + e ig ; E[e ig e jg ] = ρσ 2 e ρ is the intra-class correlation coefficient; if e ig = v g + η ig, ρ = σ2 v σ 2 v + σ 2 η The Moulton factor (Moulton 1986) is the square root of V ( ˆβ 1 ) V OLS ( ˆβ 1 ) = 1 + (n 1)ρ n is the size of each group n = 100, ρ = 0.2, the Moulton factor is 4.56! Yiqing Xu (MIT) Rec5 March 7, / 16

8 Clustering Fix Liang and Zeger (1986) ˆΩ cl = (X X ) 1 ( g X g ˆΨ g X g ) (X X ) 1 Ψ g = αê g ê g in which α adjusts degrees of freedom Same old, same old: clustered standard errors is downward biased with few clusters Try regression of group averages to test robustness Ȳ g = β 0 + β 1 x g + ē g (1) actual normality; (2) covariate-adjustment Block bootstrap Yiqing Xu (MIT) Rec5 March 7, / 16

9 Bootstrap Theory Suppose W = (W 1,, W n ) is an i.i.d. random sample of the distribution F We are interested in θ(f ) We consider an estimator, ˆθ = ˆθ(ˆF n ) ˆF n is the empirical distribution, ˆF (w) = 1 n n j=1 I (W j w) n, ˆF n F and ˆθ d θ(f ) (in Asymptopia!) How does it work? Generate bootstrap samples W (non-parametric; parametric) e.g. y i = x i β + ɛ i ; sampling{x i, y i } or {ˆɛ i } Calculate the bootstrap replications ˆθ(W ) Summarize the bootstrap replications Often doing so by blocks Yiqing Xu (MIT) Rec5 March 7, / 16

10 Inference Inference in Asymptopia (and with Weak Null) Robust standard error and biases Clustering Bootstrap Permutation inference (and with Sharp Null) Yiqing Xu (MIT) Rec5 March 7, / 16

11 Permutation Inference - Motivations Benefits: No longer in Aymptopia No needs for limiting theories Costs: Research money on buying more powerful computers Sharp Null Sharp Null p-values in the absence of S.E. Minimal assumptions (not even SUTVA, why?) Applicable when things getting complicated Yiqing Xu (MIT) Rec5 March 7, / 16

12 Summary Take-away Robust S.E. has finite sample bias; can be even worse then the conventional HC 1 -HC 3 helps but no panacea Clustering is huge, but clustered S.E. also is downward-biased Block bootstrap improves things but we re not sure If you re tired of the world of Aymptopia come back to real world and try permutation inference, if (1) you have powerful computers (2) you enjoy explaining stuff to people Yiqing Xu (MIT) Rec5 March 7, / 16

13 1 Inference of Frequentists 2 Power Calculations

14 The Logic of Power Calculations Y i = γ + δd i + ɛ i, i = 1,, N, ɛ i N(0, σ 2 ɛ ) From data structure to Minimum Detectable Effect (MDE) You know σ ɛ (from previous research, or your gut) You know N You know the Null hypothesis, e.g., δ = 0 You calculate SE(ˆδ), CI (ˆδ) You know the significance level you want, e.g., α = 0.05 (Type I error) MDE(δ) From a presumed effect to sample size, and ultimately, money Yiqing Xu (MIT) Rec5 March 7, / 16

15 f µ^

16 The Logic of Power Calculations Y i = γ + δd i + ɛ i, i = 1,, N, ɛ i N(0, σ 2 ɛ ) From data structure to Minimum Detectable Effect (MDE) From a presumed effect to sample size, and ultimately, money You know σ ɛ You know the size of the effect δ You know the sampling distribution of ˆδ, F (ˆδ) (1) You know N, you calculate power (2) You know how much power you want, e.g., β = 0.9 (Type II error), You calculate N, which leads to $ Yiqing Xu (MIT) Rec5 March 7, / 16

17 f µ^

18 The Logic of Power Calculations Y i = γ + δd i + ɛ i, i = 1,, N, ɛ i N(0, σ 2 ɛ ) From data structure to Minimum Detectable Effect (MDE) You know σ ɛ (from previous research, or your gut) You know N You know the Null hypothesis, e.g., δ = 0 You calculate SE(ˆδ), CI (ˆδ) You know the significance level you want, e.g., α = 0.05 (Type I error) MDE(δ) From a presumed effect to sample size, and ultimately, money You know σ ɛ You know the size of the effect δ You know the sampling distribution of ˆδ, F (ˆδ) (1) You know N, you calculate power (2) You know how much power you want, e.g., β = 0.9 (Type II error), You calculate N, which leads to $ Yiqing Xu (MIT) Rec5 March 7, / 16

19 In Summary Inference of frequentists Inference in Aymptopia ( robust, clustering, bootstrap) Permutation inference Power calculations From data structure to MDE From power to sample size and money Yiqing Xu (MIT) Rec5 March 7, / 16