Nonparametric Inference via Bootstrapping the Debiased Estimator
|
|
- Maximilian Higgins
- 5 years ago
- Views:
Transcription
1 Nonparametric Inference via Bootstrapping the Debiased Estimator Yen-Chi Chen Department of Statistics, University of Washington ICSA-Canada Chapter Symposium / 21
2 Problem Setup Let X 1,, X n be an IID random sample from an unknown distribution function with a density function p. For simplicity, we assume p is supported on [0, 1] d. Goal: given a level α, we want to find L α (x), U α (x) using the random sample such that ( P L α (x) p(x) U α (x) x [0, 1] d) 1 α + o(1). Namely, [L α (x), U α (x)] forms an asymptotic simultaneous confidence band of p(x). 2 / 21
3 Simple Approach: using the KDE A classical approach is to construct L α (x), U α (x) using the kernel density estimator (KDE). Let p h (x) = 1 nh d n ( ) Xi x K h i=1 be the KDE where h > 0 is the smoothing bandwidth and K(x) is a smooth function such as a Gaussian. We pick t α such that L α (x) = p h (x) t α, U α (x) = p h (x) + t α. As long as we choose t α wisely, the resulting confidence band is asymptotically valid. 3 / 21
4 Simple Approach: the L Error How do we choose t α to obtain a valid confidence band? A simple idea: inverting the L error. 4 / 21
5 Simple Approach: the L Error How do we choose t α to obtain a valid confidence band? A simple idea: inverting the L error. Let F n (t) be the CDF of p h p = sup x p h (x) p(x). Then the value tα = Fn 1 (1 α) has a nice property: P( p h p t α) = 1 α. 4 / 21
6 Simple Approach: the L Error How do we choose t α to obtain a valid confidence band? A simple idea: inverting the L error. Let F n (t) be the CDF of p h p = sup x p h (x) p(x). Then the value tα = Fn 1 (1 α) has a nice property: This implies Thus, P( p h p t α) = 1 α. P( p h (x) p(x) t α x [0, 1] d ) = 1 α. L α(x) = p h (x) t α, U α(x) = p h (x) + t α leads to a simultaneous confidence band. 4 / 21
7 Simple Approach: the Bootstrap - 1 The previous method is great it works even in a finite sample case. However, it has a critical problem: we do not know the distribution F n! So we cannot compute the quantile. 5 / 21
8 Simple Approach: the Bootstrap - 1 The previous method is great it works even in a finite sample case. However, it has a critical problem: we do not know the distribution F n! So we cannot compute the quantile. A simple solution: using the bootstrap (we will use the empirical bootstrap). 5 / 21
9 Simple Approach: the Bootstrap - 2 Let X1,, X n be a bootstrap sample. We first compute the bootstrap KDE: p h (x) = 1 n ( X nh d K i i=1 x h ). Then we compute the bootstrap L error W = p h p h. After repeating the bootstrap procedure B times, we obtain realizations W 1,, W B. Compute the empirical CDF F n (t) = 1 B B I (W l t). l=1 Finally, we use t α = F 1 n (1 α) and construct the confidence band as L α(x) = p h (x) t α, Ûα(x) = p h (x) + t α. 6 / 21
10 Simple Approach: the Bootstrap - 3 Does the bootstrap approach work? 7 / 21
11 Simple Approach: the Bootstrap - 3 Does the bootstrap approach work? It depends. The bootstrap works if in the sense that p h p h p h p sup P( p h p h < t) P( p h p < t) = o(1). t 7 / 21
12 Simple Approach: the Bootstrap - 3 Does the bootstrap approach work? It depends. The bootstrap works if in the sense that p h p h p h p sup P( p h p h < t) P( p h p < t) = o(1). t However, the above bound holds if we undersmooth the data (Neumann and Polzehl 1998, Chernozhukov et al. 2014). Namely, we choose the smoothing bandwidth h = o(n 1 4+d ). 7 / 21
13 Simple Approach: the Bootstrap - 4 Why do we need to undersmooth the data? 8 / 21
14 Simple Approach: the Bootstrap - 4 Why do we need to undersmooth the data? The L error has a bias-variance tradeoff: ( ) log n p h p = O(h 2 ) +O }{{} P nh d. Bias }{{} stochastic error 8 / 21
15 Simple Approach: the Bootstrap - 4 Why do we need to undersmooth the data? The L error has a bias-variance tradeoff: ( ) log n p h p = O(h 2 ) +O }{{} P nh d. Bias }{{} stochastic error The bootstrap L error is capable of capturing the errors in the stochastic part. However, it does not capture the bias. 8 / 21
16 Simple Approach: the Bootstrap - 4 Why do we need to undersmooth the data? The L error has a bias-variance tradeoff: ( ) log n p h p = O(h 2 ) +O }{{} P nh d. Bias }{{} stochastic error The bootstrap L error is capable of capturing the errors in the stochastic part. However, it does not capture the bias. Undersmooth guarantees that the bias is of a smaller order so we can ignore it. 8 / 21
17 Problem of Undersmoothing ( ) log n p h p = O(h 2 ) + O }{{} P nh d. Bias }{{} stochastic error Undermoothing has a problem: we do not have the optimal convergence rate. The optimal rate occurs when we balance the bias and stochastic error: h = h opt n 1 d+4 (ignoring the log n factor). 9 / 21
18 Problem of Undersmoothing ( ) log n p h p = O(h 2 ) + O }{{} P nh d. Bias }{{} stochastic error Undermoothing has a problem: we do not have the optimal convergence rate. The optimal rate occurs when we balance the bias and stochastic error: h = h opt n 1 d+4 (ignoring the log n factor). A remedy to this problem: choose h optimally but correct the bias (debiased method). 9 / 21
19 The Debiased Method - 1 The idea of the debiased method is based on the fact that a leading term of O(h 2 ) is h 2 2 C K 2 p(x), where C K is a known constant depending on the kernel function and 2 is the Laplacian operator. 10 / 21
20 The Debiased Method - 1 The idea of the debiased method is based on the fact that a leading term of O(h 2 ) is h 2 2 C K 2 p(x), where C K is a known constant depending on the kernel function and 2 is the Laplacian operator. We can estimate 2 p via applying the Laplacian operator to a KDE p h. However, such an estimator is inconsistent when we choose h opt n 1 d+4 because 2 p h (x) 2 p(x) = O(h 2 ) + O P ( 1 nh d+4 ). 10 / 21
21 The Debiased Method - 1 The idea of the debiased method is based on the fact that a leading term of O(h 2 ) is h 2 2 C K 2 p(x), where C K is a known constant depending on the kernel function and 2 is the Laplacian operator. We can estimate 2 p via applying the Laplacian operator to a KDE p h. However, such an estimator is inconsistent when we choose h opt n 1 d+4 because 2 p h (x) 2 p(x) = O(h 2 ) + O P ( 1 nh d+4 The choice h = h opt n 1 d+4 implies 2 p h (x) 2 p(x) = o(1) + O P (1). ). 10 / 21
22 The Debiased Method - 2 To handle this situation, people suggested using two KDE s, one for estimating the density and the other for estimating the bias. 1 This idea has been used in Calonico et al. (2015) for a pointwise confidence interval. 11 / 21
23 The Debiased Method - 2 To handle this situation, people suggested using two KDE s, one for estimating the density and the other for estimating the bias. However, actually we ONLY need one KDE. 1 This idea has been used in Calonico et al. (2015) for a pointwise confidence interval. 11 / 21
24 The Debiased Method - 2 To handle this situation, people suggested using two KDE s, one for estimating the density and the other for estimating the bias. However, actually we ONLY need one KDE. We propose using the same KDE p h (x) to debias the estimator 1. 1 This idea has been used in Calonico et al. (2015) for a pointwise confidence interval. 11 / 21
25 The Debiased Method - 2 To handle this situation, people suggested using two KDE s, one for estimating the density and the other for estimating the bias. However, actually we ONLY need one KDE. We propose using the same KDE p h (x) to debias the estimator 1. Namely, we propose to use with h = h opt n 1 d+4. p h (x) = p h (x) h2 2 C K 2 p h (x) The estimator p h (x) is called the debiased estimator. 1 This idea has been used in Calonico et al. (2015) for a pointwise confidence interval. 11 / 21
26 The Debiased Method + Bootstrap To construct a confidence band, we use the bootstrap again but this time we compute the bootstrap debiased estimator and evaluate p h p h. p h (x) = p h2 h (x) 2 C K 2 p h (x) After repeating the bootstrap procedure many times, we compute the EDF F n of the realizations of p h p h and obtain the quantile t α = (1 α). The confidence band is F 1 n L α (x) = p h (x) t α, Ũ α (x) = p h (x) + t α. 12 / 21
27 Theory of the Debiased Method Theorem (Chen 2017) Assume p belongs to β-hölder class with β > 2 and the kernel function satisfies smoothness conditions. When h n 1 d+4, P ( Lα (x) p(x) Ũα(x) x [0, 1] d) = 1 α + o(1). Namely, the debiased estimator leads to an asymptotic simultaneous confidence band under the choice h n 1 d / 21
28 Why the Debiased Method Work? - 1 Why the debiased method work? Didn t we have an inconsistent bias estimator? 14 / 21
29 Why the Debiased Method Work? - 1 Why the debiased method work? Didn t we have an inconsistent bias estimator? We indeed do not have a consistent bias estimator but this is fine! 14 / 21
30 Why the Debiased Method Work? - 1 Why the debiased method work? Didn t we have an inconsistent bias estimator? We indeed do not have a consistent bias estimator but this is fine! Recall that when h n 1 d+4, 2 p h (x) 2 p(x) = o(1) + }{{} O P (1) }{{}. bias stochastic variation 14 / 21
31 Why the Debiased Method Work? - 1 Why the debiased method work? Didn t we have an inconsistent bias estimator? We indeed do not have a consistent bias estimator but this is fine! Recall that when h n 1 d+4, 2 p h (x) 2 p(x) = o(1) + }{{} O P (1) }{{}. bias stochastic variation Thus, our debiased estimator has three errors: p h (x) p(x) = p h (x) h2 2 C K p h (x) p(x) = h2 ( 1 2 C K 2 p(x) + o(h 2 ) +O P }{{} nh d bias ) h2 2 C K 2 p h (x) 14 / 21
32 Why the Debiased Method Work? - 2 The above equation equals (h n 1 d+4 ) ( ) p h (x) p(x) = h2 1 2 C K p(x) + o(h 2 ) +O P }{{} nh d h2 2 C K p h (x) bias = o(h 2 ) + O P ( 1 nh d ) + h2 2 C ( K 2 p(x) 2 p h (x) ) }{{} =o(1)+o P (1) ( ) 1 = o(h 2 ) + O P nh d + o(h 2 ) + O P (h 2 ) ( ) 1 = o(h 2 ( ) + O P nh d + O P h 2 ). Both the orange and purple terms are stochastic variation. Orange: from estimating the density. Purple: from estimating the bias. 15 / 21
33 Why the Debiased Method Work? - 3 When h n 1 d+4, the error rate ( ) 1 p h (x) p(x) = o(h 2 ( ) + O P nh d + O P h 2 ) = O P (n 2 d+4 ) is dominated by the stochastic variation. As a result, the bootstrap can capture the errors, leading to an asymptotic valid confidence band. 16 / 21
34 Why the Debiased Method Work? - 4 Actually, after closely inspecting the debiased estimator, you can find that p h (x) = p h (x) h2 2 C K 2 p h (x) = 1 n ( ) Xi x nh d K h2 h 2 C 1 K nh d i=1 = 1 n ( ) Xi x nh d M, h i=1 n ( ) 2 Xi x K h where M(x) = K(x) C K 2 2 K(x). Namely, the debiased estimator is a KDE with kernel function M(x)! i=1 17 / 21
35 Why the Debiased Method Work? - 5 The kernel function M(x) = K(x) C K 2 2 K(x) is actually a higher order kernel. 18 / 21
36 Why the Debiased Method Work? - 5 The kernel function M(x) = K(x) C K 2 2 K(x) is actually a higher order kernel. You can show that if the kernel function K(x) is a γ-th order kernel function, then the corresponding M(x) will be a (γ + 2)-th order kernel (Calonico et al. 2015, Scott 2015). 18 / 21
37 Why the Debiased Method Work? - 5 The kernel function M(x) = K(x) C K 2 2 K(x) is actually a higher order kernel. You can show that if the kernel function K(x) is a γ-th order kernel function, then the corresponding M(x) will be a (γ + 2)-th order kernel (Calonico et al. 2015, Scott 2015). Because the debiased estimator p h (x) uses a higher order kernel, the bias is moved to the next order, leaving the stochastic variation dominating the error. 18 / 21
38 Simulation Density Estimation, 95% CI Confidence Sets, Gaussian, N= Original KDE Debiased KDE Bootstrap Coverage Original KDE Debiased KDE X Nominal Coverage 19 / 21
39 Conclusion We illustrate a bootstrap approach to construct a simultaneous confidence band via a debiased KDE. This approach allows us to choose the smoothing bandwidth optimally and still leads to an asymptotic confidence band. A similar idea can also be applied to regression problem and local polynomial estimator. More details can be found in Chen, Yen-Chi. "Nonparametric Inference via Bootstrapping the Debiased Estimator." arxiv preprint arxiv: (2017). 20 / 21
40 Thank you! 21 / 21
41 References 1 Y.-C. Chen. Nonparametric Inference via Bootstrapping the Debiased Estimator. arxiv preprint arxiv: , Y.-C. Chen. A Tutorial on Kernel Density Estimation and Recent Advances. arxiv preprint arxiv: , S. Calonico, M. D. Cattaneo, and M. H. Farrell. On the effect of bias estimation on coverage accuracy in nonparametric inference. arxiv preprint arxiv: , V. Chernozhukov, D. Chetverikov, and K. Kato. Anti-concentration and honest, adaptive confidence bands. The Annals of Statistics, 42(5): , M. H. Neumann and J. Polzehl. Simultaneous bootstrap confidence bands in nonparametric regression. Journal of Nonparametric Statistics, 9(4): , D. W. Scott. Multivariate density estimation: theory, practice, and visualization. John Wiley & Sons, / 21
Confidence intervals for kernel density estimation
Stata User Group - 9th UK meeting - 19/20 May 2003 Confidence intervals for kernel density estimation Carlo Fiorio c.fiorio@lse.ac.uk London School of Economics and STICERD Stata User Group - 9th UK meeting
More informationarxiv: v2 [stat.me] 12 Sep 2017
1 arxiv:1704.03924v2 [stat.me] 12 Sep 2017 A Tutorial on Kernel Density Estimation and Recent Advances Yen-Chi Chen Department of Statistics University of Washington September 13, 2017 This tutorial provides
More informationStatistica Sinica Preprint No: SS
Statistica Sinica Preprint No: SS-017-0013 Title A Bootstrap Method for Constructing Pointwise and Uniform Confidence Bands for Conditional Quantile Functions Manuscript ID SS-017-0013 URL http://wwwstatsinicaedutw/statistica/
More informationOptimal Ridge Detection using Coverage Risk
Optimal Ridge Detection using Coverage Risk Yen-Chi Chen Department of Statistics Carnegie Mellon University yenchic@andrew.cmu.edu Shirley Ho Department of Physics Carnegie Mellon University shirleyh@andrew.cmu.edu
More informationSimple and Honest Confidence Intervals in Nonparametric Regression
Simple and Honest Confidence Intervals in Nonparametric Regression Timothy B. Armstrong Yale University Michal Kolesár Princeton University June, 206 Abstract We consider the problem of constructing honest
More informationStatistics: Learning models from data
DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial
More informationModel-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego
Model-free prediction intervals for regression and autoregression Dimitris N. Politis University of California, San Diego To explain or to predict? Models are indispensable for exploring/utilizing relationships
More informationInference in Nonparametric Series Estimation with Data-Dependent Number of Series Terms
Inference in Nonparametric Series Estimation with Data-Dependent Number of Series Terms Byunghoon ang Department of Economics, University of Wisconsin-Madison First version December 9, 204; Revised November
More informationLecture 2: CDF and EDF
STAT 425: Introduction to Nonparametric Statistics Winter 2018 Instructor: Yen-Chi Chen Lecture 2: CDF and EDF 2.1 CDF: Cumulative Distribution Function For a random variable X, its CDF F () contains all
More informationStatistical inference on Lévy processes
Alberto Coca Cabrero University of Cambridge - CCA Supervisors: Dr. Richard Nickl and Professor L.C.G.Rogers Funded by Fundación Mutua Madrileña and EPSRC MASDOC/CCA student workshop 2013 26th March Outline
More informationSTAT Section 2.1: Basic Inference. Basic Definitions
STAT 518 --- Section 2.1: Basic Inference Basic Definitions Population: The collection of all the individuals of interest. This collection may be or even. Sample: A collection of elements of the population.
More informationSIMPLE AND HONEST CONFIDENCE INTERVALS IN NONPARAMETRIC REGRESSION. Timothy B. Armstrong and Michal Kolesár. June 2016 Revised October 2016
SIMPLE AND HONEST CONFIDENCE INTERVALS IN NONPARAMETRIC REGRESSION By Timothy B. Armstrong and Michal Kolesár June 2016 Revised October 2016 COWLES FOUNDATION DISCUSSION PAPER NO. 2044R COWLES FOUNDATION
More informationoptimal inference in a class of nonparametric models
optimal inference in a class of nonparametric models Timothy Armstrong (Yale University) Michal Kolesár (Princeton University) September 2015 setup Interested in inference on linear functional Lf in regression
More informationA Simple Adjustment for Bandwidth Snooping
A Simple Adjustment for Bandwidth Snooping Timothy B. Armstrong Yale University Michal Kolesár Princeton University June 28, 2017 Abstract Kernel-based estimators such as local polynomial estimators in
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationSmooth nonparametric estimation of a quantile function under right censoring using beta kernels
Smooth nonparametric estimation of a quantile function under right censoring using beta kernels Chanseok Park 1 Department of Mathematical Sciences, Clemson University, Clemson, SC 29634 Short Title: Smooth
More informationTest for Discontinuities in Nonparametric Regression
Communications of the Korean Statistical Society Vol. 15, No. 5, 2008, pp. 709 717 Test for Discontinuities in Nonparametric Regression Dongryeon Park 1) Abstract The difference of two one-sided kernel
More informationBickel Rosenblatt test
University of Latvia 28.05.2011. A classical Let X 1,..., X n be i.i.d. random variables with a continuous probability density function f. Consider a simple hypothesis H 0 : f = f 0 with a significance
More informationSMOOTHED BLOCK EMPIRICAL LIKELIHOOD FOR QUANTILES OF WEAKLY DEPENDENT PROCESSES
Statistica Sinica 19 (2009), 71-81 SMOOTHED BLOCK EMPIRICAL LIKELIHOOD FOR QUANTILES OF WEAKLY DEPENDENT PROCESSES Song Xi Chen 1,2 and Chiu Min Wong 3 1 Iowa State University, 2 Peking University and
More informationInference on distributions and quantiles using a finite-sample Dirichlet process
Dirichlet IDEAL Theory/methods Simulations Inference on distributions and quantiles using a finite-sample Dirichlet process David M. Kaplan University of Missouri Matt Goldman UC San Diego Midwest Econometrics
More informationA Simple Adjustment for Bandwidth Snooping
A Simple Adjustment for Bandwidth Snooping Timothy B. Armstrong Yale University Michal Kolesár Princeton University October 18, 2016 Abstract Kernel-based estimators are often evaluated at multiple bandwidths
More informationIV Quantile Regression for Group-level Treatments, with an Application to the Distributional Effects of Trade
IV Quantile Regression for Group-level Treatments, with an Application to the Distributional Effects of Trade Denis Chetverikov Brad Larsen Christopher Palmer UCLA, Stanford and NBER, UC Berkeley September
More informationA Novel Nonparametric Density Estimator
A Novel Nonparametric Density Estimator Z. I. Botev The University of Queensland Australia Abstract We present a novel nonparametric density estimator and a new data-driven bandwidth selection method with
More informationSmooth simultaneous confidence bands for cumulative distribution functions
Journal of Nonparametric Statistics, 2013 Vol. 25, No. 2, 395 407, http://dx.doi.org/10.1080/10485252.2012.759219 Smooth simultaneous confidence bands for cumulative distribution functions Jiangyan Wang
More informationOn variable bandwidth kernel density estimation
JSM 04 - Section on Nonparametric Statistics On variable bandwidth kernel density estimation Janet Nakarmi Hailin Sang Abstract In this paper we study the ideal variable bandwidth kernel estimator introduced
More informationTime Series and Forecasting Lecture 4 NonLinear Time Series
Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations
More informationNonparametric Function Estimation with Infinite-Order Kernels
Nonparametric Function Estimation with Infinite-Order Kernels Arthur Berg Department of Statistics, University of Florida March 15, 2008 Kernel Density Estimation (IID Case) Let X 1,..., X n iid density
More informationfinite-sample optimal estimation and inference on average treatment effects under unconfoundedness
finite-sample optimal estimation and inference on average treatment effects under unconfoundedness Timothy Armstrong (Yale University) Michal Kolesár (Princeton University) September 2017 Introduction
More informationOn the Effect of Bias Estimation on Coverage Accuracy in Nonparametric Inference
On the Effect of Bias Estimation on Coverage Accuracy in Nonparametric Inference Sebastian Calonico Matias D. Cattaneo Max H. Farrell November 4, 014 PRELIMINARY AND INCOMPLETE COMMENTS WELCOME Abstract
More informationSIMPLE AND HONEST CONFIDENCE INTERVALS IN NONPARAMETRIC REGRESSION. Timothy B. Armstrong and Michal Kolesár. June 2016 Revised March 2018
SIMPLE AND HONEST CONFIDENCE INTERVALS IN NONPARAMETRIC REGRESSION By Timothy B. Armstrong and Michal Kolesár June 2016 Revised March 2018 COWLES FOUNDATION DISCUSSION PAPER NO. 2044R2 COWLES FOUNDATION
More informationPreface. 1 Nonparametric Density Estimation and Testing. 1.1 Introduction. 1.2 Univariate Density Estimation
Preface Nonparametric econometrics has become one of the most important sub-fields in modern econometrics. The primary goal of this lecture note is to introduce various nonparametric and semiparametric
More informationSelective Inference for Effect Modification
Inference for Modification (Joint work with Dylan Small and Ashkan Ertefaie) Department of Statistics, University of Pennsylvania May 24, ACIC 2017 Manuscript and slides are available at http://www-stat.wharton.upenn.edu/~qyzhao/.
More informationCS 147: Computer Systems Performance Analysis
CS 147: Computer Systems Performance Analysis Summarizing Variability and Determining Distributions CS 147: Computer Systems Performance Analysis Summarizing Variability and Determining Distributions 1
More informationQuantile Regression for Extraordinarily Large Data
Quantile Regression for Extraordinarily Large Data Shih-Kang Chao Department of Statistics Purdue University November, 2016 A joint work with Stanislav Volgushev and Guang Cheng Quantile regression Two-step
More informationAdaptivity to Local Smoothness and Dimension in Kernel Regression
Adaptivity to Local Smoothness and Dimension in Kernel Regression Samory Kpotufe Toyota Technological Institute-Chicago samory@tticedu Vikas K Garg Toyota Technological Institute-Chicago vkg@tticedu Abstract
More informationSemi-Nonparametric Inferences for Massive Data
Semi-Nonparametric Inferences for Massive Data Guang Cheng 1 Department of Statistics Purdue University Statistics Seminar at NCSU October, 2015 1 Acknowledge NSF, Simons Foundation and ONR. A Joint Work
More informationNonparametric Regression
Nonparametric Regression Econ 674 Purdue University April 8, 2009 Justin L. Tobias (Purdue) Nonparametric Regression April 8, 2009 1 / 31 Consider the univariate nonparametric regression model: where y
More informationQuantile Processes for Semi and Nonparametric Regression
Quantile Processes for Semi and Nonparametric Regression Shih-Kang Chao Department of Statistics Purdue University IMS-APRM 2016 A joint work with Stanislav Volgushev and Guang Cheng Quantile Response
More informationGeneralized Multivariate Rank Type Test Statistics via Spatial U-Quantiles
Generalized Multivariate Rank Type Test Statistics via Spatial U-Quantiles Weihua Zhou 1 University of North Carolina at Charlotte and Robert Serfling 2 University of Texas at Dallas Final revision for
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2
MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2 1 Bootstrapped Bias and CIs Given a multiple regression model with mean and
More informationMinimax Estimation of a nonlinear functional on a structured high-dimensional model
Minimax Estimation of a nonlinear functional on a structured high-dimensional model Eric Tchetgen Tchetgen Professor of Biostatistics and Epidemiologic Methods, Harvard U. (Minimax ) 1 / 38 Outline Heuristics
More informationMultiscale Adaptive Inference on Conditional Moment Inequalities
Multiscale Adaptive Inference on Conditional Moment Inequalities Timothy B. Armstrong 1 Hock Peng Chan 2 1 Yale University 2 National University of Singapore June 2013 Conditional moment inequality models
More informationDefect Detection using Nonparametric Regression
Defect Detection using Nonparametric Regression Siana Halim Industrial Engineering Department-Petra Christian University Siwalankerto 121-131 Surabaya- Indonesia halim@petra.ac.id Abstract: To compare
More informationIntegral approximation by kernel smoothing
Integral approximation by kernel smoothing François Portier Université catholique de Louvain - ISBA August, 29 2014 In collaboration with Bernard Delyon Topic of the talk: Given ϕ : R d R, estimation of
More informationUniform Post Selection Inference for LAD Regression and Other Z-estimation problems. ArXiv: Alexandre Belloni (Duke) + Kengo Kato (Tokyo)
Uniform Post Selection Inference for LAD Regression and Other Z-estimation problems. ArXiv: 1304.0282 Victor MIT, Economics + Center for Statistics Co-authors: Alexandre Belloni (Duke) + Kengo Kato (Tokyo)
More informationInference For High Dimensional M-estimates. Fixed Design Results
: Fixed Design Results Lihua Lei Advisors: Peter J. Bickel, Michael I. Jordan joint work with Peter J. Bickel and Noureddine El Karoui Dec. 8, 2016 1/57 Table of Contents 1 Background 2 Main Results and
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationNonparametric Density Estimation
Nonparametric Density Estimation Econ 690 Purdue University Justin L. Tobias (Purdue) Nonparametric Density Estimation 1 / 29 Density Estimation Suppose that you had some data, say on wages, and you wanted
More information12 - Nonparametric Density Estimation
ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6
More informationMachine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.
Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted
More informationProgram Evaluation with High-Dimensional Data
Program Evaluation with High-Dimensional Data Alexandre Belloni Duke Victor Chernozhukov MIT Iván Fernández-Val BU Christian Hansen Booth ESWC 215 August 17, 215 Introduction Goal is to perform inference
More informationThe Bootstrap: Theory and Applications. Biing-Shen Kuo National Chengchi University
The Bootstrap: Theory and Applications Biing-Shen Kuo National Chengchi University Motivation: Poor Asymptotic Approximation Most of statistical inference relies on asymptotic theory. Motivation: Poor
More informationUniform Inference for Conditional Factor Models with Instrumental and Idiosyncratic Betas
Uniform Inference for Conditional Factor Models with Instrumental and Idiosyncratic Betas Yuan Xiye Yang Rutgers University Dec 27 Greater NY Econometrics Overview main results Introduction Consider a
More informationLarge Sample Properties of Partitioning-Based Series Estimators
Large Sample Properties of Partitioning-Based Series Estimators Matias D. Cattaneo Max H. Farrell Yingjie Feng April 13, 2018 Abstract We present large sample results for partitioning-based least squares
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More informationThe Intersection of Statistics and Topology:
The Intersection of Statistics and Topology: Confidence Sets for Persistence Diagrams Brittany Terese Fasy joint work with S. Balakrishnan, F. Lecci, A. Rinaldo, A. Singh, L. Wasserman 3 June 2014 [FLRWBS]
More informationInference in Regression Discontinuity Designs with a Discrete Running Variable
Inference in Regression Discontinuity Designs with a Discrete Running Variable Michal Kolesár Christoph Rothe arxiv:1606.04086v4 [stat.ap] 18 Nov 2017 November 21, 2017 Abstract We consider inference in
More informationNonparametric Heterogeneity Testing for Massive Data
Nonparametric Heterogeneity Testing for Massive Data Abstract A massive dataset often consists of a growing number of potentially heterogeneous sub-populations. This paper is concerned about testing various
More informationLocally Robust Semiparametric Estimation
Locally Robust Semiparametric Estimation Victor Chernozhukov Juan Carlos Escanciano Hidehiko Ichimura Whitney K. Newey The Institute for Fiscal Studies Department of Economics, UCL cemmap working paper
More informationSupplemental Material for KERNEL-BASED INFERENCE IN TIME-VARYING COEFFICIENT COINTEGRATING REGRESSION. September 2017
Supplemental Material for KERNEL-BASED INFERENCE IN TIME-VARYING COEFFICIENT COINTEGRATING REGRESSION By Degui Li, Peter C. B. Phillips, and Jiti Gao September 017 COWLES FOUNDATION DISCUSSION PAPER NO.
More informationLearning 2 -Continuous Regression Functionals via Regularized Riesz Representers
Learning 2 -Continuous Regression Functionals via Regularized Riesz Representers Victor Chernozhukov MIT Whitney K. Newey MIT Rahul Singh MIT September 10, 2018 Abstract Many objects of interest can be
More informationA simple bootstrap method for constructing nonparametric confidence bands for functions
A simple bootstrap method for constructing nonparametric confidence bands for functions Peter Hall Joel Horowitz The Institute for Fiscal Studies Department of Economics, UCL cemmap working paper CWP4/2
More informationarxiv: v1 [stat.me] 7 Jun 2015
Optimal Ridge Detection using Coverage Risk arxiv:156.78v1 [stat.me] 7 Jun 15 Yen-Chi Chen Department of Statistics Carnegie Mellon University yenchic@andrew.cmu.edu Shirley Ho Department of Physics Carnegie
More informationThe assumptions are needed to give us... valid standard errors valid confidence intervals valid hypothesis tests and p-values
Statistical Consulting Topics The Bootstrap... The bootstrap is a computer-based method for assigning measures of accuracy to statistical estimates. (Efron and Tibshrani, 1998.) What do we do when our
More informationDensity Estimation. We are concerned more here with the non-parametric case (see Roger Barlow s lectures for parametric statistics)
Density Estimation Density Estimation: Deals with the problem of estimating probability density functions (PDFs) based on some data sampled from the PDF. May use assumed forms of the distribution, parameterized
More informationEMPIRICAL EVALUATION OF DATA-BASED DENSITY ESTIMATION
Proceedings of the 2006 Winter Simulation Conference L. F. Perrone, F. P. Wieland, J. Liu, B. G. Lawson, D. M. Nicol, and R. M. Fujimoto, eds. EMPIRICAL EVALUATION OF DATA-BASED DENSITY ESTIMATION E. Jack
More informationFrom CDF to PDF A Density Estimation Method for High Dimensional Data
From CDF to PDF A Density Estimation Method for High Dimensional Data Shengdong Zhang Simon Fraser University sza75@sfu.ca arxiv:1804.05316v1 [stat.ml] 15 Apr 2018 April 17, 2018 1 Introduction Probability
More information7 Influence Functions
7 Influence Functions The influence function is used to approximate the standard error of a plug-in estimator. The formal definition is as follows. 7.1 Definition. The Gâteaux derivative of T at F in the
More informationOn Fractile Transformation of Covariates in Regression 1
1 / 13 On Fractile Transformation of Covariates in Regression 1 Bodhisattva Sen Department of Statistics Columbia University, New York ERCIM 10 11 December, 2010 1 Joint work with Probal Chaudhuri, Indian
More informationNonparametric confidence intervals. for receiver operating characteristic curves
Nonparametric confidence intervals for receiver operating characteristic curves Peter G. Hall 1, Rob J. Hyndman 2, and Yanan Fan 3 5 December 2003 Abstract: We study methods for constructing confidence
More informationOPTIMAL INFERENCE IN A CLASS OF REGRESSION MODELS. Timothy B. Armstrong and Michal Kolesár. May 2016 Revised May 2017
OPTIMAL INFERENCE IN A CLASS OF REGRESSION MODELS By Timothy B. Armstrong and Michal Kolesár May 2016 Revised May 2017 COWLES FOUNDATION DISCUSSION PAPER NO. 2043R COWLES FOUNDATION FOR RESEARCH IN ECONOMICS
More informationExploiting k-nearest Neighbor Information with Many Data
Exploiting k-nearest Neighbor Information with Many Data 2017 TEST TECHNOLOGY WORKSHOP 2017. 10. 24 (Tue.) Yung-Kyun Noh Robotics Lab., Contents Nonparametric methods for estimating density functions Nearest
More informationChapter 9. Non-Parametric Density Function Estimation
9-1 Density Estimation Version 1.2 Chapter 9 Non-Parametric Density Function Estimation 9.1. Introduction We have discussed several estimation techniques: method of moments, maximum likelihood, and least
More informationSTATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
More informationModal Regression using Kernel Density Estimation: a Review
Modal Regression using Kernel Density Estimation: a Review Yen-Chi Chen arxiv:1710.07004v2 [stat.me] 7 Dec 2017 Abstract We review recent advances in modal regression studies using kernel density estimation.
More informationCoverage Error Optimal Confidence Intervals
Coverage Error Optimal Confidence Intervals Sebastian Calonico Matias D. Cattaneo Max H. Farrell August 3, 2018 Abstract We propose a framework for ranking confidence interval estimators in terms of their
More informationNonparametric Regression. Changliang Zou
Nonparametric Regression Institute of Statistics, Nankai University Email: nk.chlzou@gmail.com Smoothing parameter selection An overall measure of how well m h (x) performs in estimating m(x) over x (0,
More informationNonparametric Methods
Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis
More informationKernel density estimation
Kernel density estimation Patrick Breheny October 18 Patrick Breheny STA 621: Nonparametric Statistics 1/34 Introduction Kernel Density Estimation We ve looked at one method for estimating density: histograms
More informationSupplement to Quantile-Based Nonparametric Inference for First-Price Auctions
Supplement to Quantile-Based Nonparametric Inference for First-Price Auctions Vadim Marmer University of British Columbia Artyom Shneyerov CIRANO, CIREQ, and Concordia University August 30, 2010 Abstract
More informationDiagnostics. Gad Kimmel
Diagnostics Gad Kimmel Outline Introduction. Bootstrap method. Cross validation. ROC plot. Introduction Motivation Estimating properties of an estimator. Given data samples say the average. x 1, x 2,...,
More informationBetter Bootstrap Confidence Intervals
by Bradley Efron University of Washington, Department of Statistics April 12, 2012 An example Suppose we wish to make inference on some parameter θ T (F ) (e.g. θ = E F X ), based on data We might suppose
More informationUnderstanding Generalization Error: Bounds and Decompositions
CIS 520: Machine Learning Spring 2018: Lecture 11 Understanding Generalization Error: Bounds and Decompositions Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the
More informationIntroduction to Regression
Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric
More informationInference in Regression Discontinuity Designs with a Discrete Running Variable
Inference in Regression Discontinuity Designs with a Discrete Running Variable Michal Kolesár Christoph Rothe June 21, 2016 Abstract We consider inference in regression discontinuity designs when the running
More informationLecture 3: Introduction to Complexity Regularization
ECE90 Spring 2007 Statistical Learning Theory Instructor: R. Nowak Lecture 3: Introduction to Complexity Regularization We ended the previous lecture with a brief discussion of overfitting. Recall that,
More informationSingle Index Quantile Regression for Heteroscedastic Data
Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University JSM, 2015 E. Christou, M. G. Akritas (PSU) SIQR JSM, 2015
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationOPTIMAL INFERENCE IN A CLASS OF REGRESSION MODELS. Timothy B. Armstrong and Michal Kolesár. May 2016 COWLES FOUNDATION DISCUSSION PAPER NO.
OPTIMAL INFERENCE IN A CLASS OF REGRESSION MODELS By Timothy B. Armstrong and Michal Kolesár May 2016 COWLES FOUNDATION DISCUSSION PAPER NO. 2043 COWLES FOUNDATION FOR RESEARCH IN ECONOMICS YALE UNIVERSITY
More informationA Simple Adjustment for Bandwidth Snooping
Review of Economic Studies (207) 0, 35 0034-6527/7/0000000$02.00 c 207 The Review of Economic Studies Limited A Simple Adjustment for Bandwidth Snooping TIMOTHY B. ARMSTRONG Yale University E-mail: timothy.armstrong@yale.edu
More informationA new method of nonparametric density estimation
A new method of nonparametric density estimation Andrey Pepelyshev Cardi December 7, 2011 1/32 A. Pepelyshev A new method of nonparametric density estimation Contents Introduction A new density estimate
More informationIDEAL Inference on Conditional Quantiles via Interpolated Duals of Exact Analytic L-statistics
IDEAL Inference on Conditional Quantiles via Interpolated Duals of Exact Analytic L-statistics David M. Kaplan Department of Economics, UC San Diego December 26, 2012; first version May 30, 2012 Abstract
More informationChapter 2: Resampling Maarten Jansen
Chapter 2: Resampling Maarten Jansen Randomization tests Randomized experiment random assignment of sample subjects to groups Example: medical experiment with control group n 1 subjects for true medicine,
More informationCross-Validation with Confidence
Cross-Validation with Confidence Jing Lei Department of Statistics, Carnegie Mellon University WHOA-PSI Workshop, St Louis, 2017 Quotes from Day 1 and Day 2 Good model or pure model? Occam s razor We really
More informationDISCUSSION: COVERAGE OF BAYESIAN CREDIBLE SETS. By Subhashis Ghosal North Carolina State University
Submitted to the Annals of Statistics DISCUSSION: COVERAGE OF BAYESIAN CREDIBLE SETS By Subhashis Ghosal North Carolina State University First I like to congratulate the authors Botond Szabó, Aad van der
More informationNONPARAMETRIC DENSITY ESTIMATION WITH RESPECT TO THE LINEX LOSS FUNCTION
NONPARAMETRIC DENSITY ESTIMATION WITH RESPECT TO THE LINEX LOSS FUNCTION R. HASHEMI, S. REZAEI AND L. AMIRI Department of Statistics, Faculty of Science, Razi University, 67149, Kermanshah, Iran. ABSTRACT
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression
More informationIndependent and conditionally independent counterfactual distributions
Independent and conditionally independent counterfactual distributions Marcin Wolski European Investment Bank M.Wolski@eib.org Society for Nonlinear Dynamics and Econometrics Tokyo March 19, 2018 Views
More informationBootstrap tuning in model choice problem
Weierstrass Institute for Applied Analysis and Stochastics Bootstrap tuning in model choice problem Vladimir Spokoiny, (with Niklas Willrich) WIAS, HU Berlin, MIPT, IITP Moscow SFB 649 Motzen, 17.07.2015
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/100 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression
More information