Nonparametric Regression
|
|
- Laurel Russell
- 5 years ago
- Views:
Transcription
1 Nonparametric Regression Econ 674 Purdue University April 8, 2009 Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
2 Consider the univariate nonparametric regression model: where y i and x i are scalars, for simplicity. Note that the marginal density for x might be obtained as: Similarly, the joint density for x and y could be estimated (simply) as: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
3 The function of interest, The denominator can be estimated, as shown on the last slide. As for the numerator, substitute in our estimator for the joint density, and obtain (assuming a symmetric, mean-zero kernel): Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
4 Thus, we have all the pieces we need to obtain a nonparametric estimator (called the Nadaraya-Watson estimator) of the conditional mean. Noting that the bandwidth terms cancel in the ratio, we obtain: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
5 Intuition Let us now try and justify this estimator in an intuitive way, much like we did for the case of nonparametric density estimation. Suppose that x is discrete-valued and we observe n 0 points with x = x 0. In this case, we might use the sample average as a consistent estimate of the conditional mean function at x 0 : This technique works great, of course, if x is discrete-valued. However, if x is continuous, the above will not work - we will never observe n 0 points for which x = x 0. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
6 Intuition To remedy this problem, we can average those y s for which the x s fall in some interval around x 0. We can then replace ˆm(x 0 ) = Ê(y x = x 0) with the sample average of the y s falling in this region: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
7 Intuition In the previous estimator, we placed equal weight on all the points in the interval, and zero weight on points outside the interval. More generally, we might replace the indicator function above with a continuous weight function: where, as before, K is a mean-zero symmetric density function. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
8 Under certain regularity conditions, we can establish pointwise consistency of the constant kernel estimator: ˆm(x 0 ) p m(x 0 ) To save time we defer the proof, although it follows similarly to the proof for the kernel density estimator. The method above generalizes to higher dimensions: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
9 Local Polynomial Regression We can consider this problem yet another way, which will lead to an estimator that improves upon the N-W-estimator. As with least-squares, we might wish to minimize the objective function: To this end, we take a second-order expansion of the regression function about the point x 0 : Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
10 Local Polynomial Regression We then substitute this expansion into our objective function and include a kernel-weighting term: From the above, the kernel-weighting term weights the points closer to x 0 more heavily than the terms farther away from x 0. (like weighted least-squares). Let α 0 = m(x 0 ), α 1 = m (x 0 ), and α 2 = m (x 0 ). Then we re-write the objective function as: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
11 Local Polynomial Regression We can stack this problem in matrix form: min α y 1 y 2. y n (x 1 (x 1 x 0 ) 1 x 0 ) 2 2 (x 1 (x 2 x 0 ) 2 x 0 ) (xn x 1 (x n x 0 ) 0 ) 2 2 α 0 α 1 α 2 ( ) x1 x K hn ( ) x2 x 0 K 0 0 hn K ( xn x0 hn ) y 1 y 2. y n which is equivalent to: (x 1 (x 1 x 0 ) 1 x 0 ) 2 2 (x 1 (x 2 x 0 ) 2 x 0 ) (x n x 0 ) (xn x 0 ) 2 2 α 0 α 1. α 2 Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
12 Local Polynomial Regression This objective function is just like the objective function which yields the GLS estimator. Thus, we have: The (1,1) element gives an estimate of the CMF at x 0, the (2,1) element gives an estimate of the marginal effect at x 0. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
13 Local Polynomial Regression So, here s the procedure for fitting a regression model via Local Polynomial regression. 1 Select a bandwidth h n and kernel K 2 Pick a set of points at which to evaluate the CMF (3 σ rule perhaps). This could also be all the x i. 3 For each point, compute ˆα as above. You can plot the (1, 1) elements to plot the conditional mean function. (Often, the (2,1) elements are of primary interest; marginal effects mimic regression coefficients.) 4 What happens if you just approximate m(x i ) with a constant α 0? Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
14 Other points to note: The above also generalizes to the multivatiate case. However, there is the curse of dimensonality - the rate of convergence slows down with the dimension of the problem: nh d n (assuming a common bandwidth is employed in all dimensions). How can you pick the bandwidth h n and the kernel K? Standard errors for the above point estimates are rather involved and difficult to compute. Bootstrapping is a possibility, though the bootstrap should correct for the bias of the estimator. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
15 It s an odd world There is a preference for odd-order fits. Let p be the order of the series expansion (ex. 1 = linear, 2 = quadratic) and v be the order of the derivative we seek to estimate. Then Ruppert and Wand (1994 Ann Stat) show that bias is reduced and performance at the boundary is improved by setting p v to be odd. This suggests a preference for local linear regression when estimating the conditional mean function. To get around the curse of dimensionality, some specify a nonparametric part for only one/few elements of x: This is called a partially linear or semilinear model. See Robinson (1988 Econometrica) or Yatchew (1997 Economics Letters) for estimation procedures. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
16 Bandwidth Selection Fan and Gijbels (1995) derive and optimal bandwidth rule. They consider an asymptotic weighted mean integrated squared error criterion: where m v (x) is the v th derivative of the CMF which we are interested in, and w(x) is a weighting function. They show the bandwidth which minimizes this criterion is of the form: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
17 Bandwidth Selection In the above, σ 2 (x) E[(y m(x)) 2 ], w(x) w 0 (x)f (x), C p,v (K) is a constant which depends on the expansion order (p), order of the derivative (v), and kernel K. Finally, m p+1 is the p + 1 th derivative of the unknown function m. This can be estimated as: [ hn ˆσ 2 1/(2p+3) w 0 (x)dx C p,v (K) n i=1 ( ˆmp+1 (x i )) 2. w 0 (x i )] We can obtain ˆσ 2 and ˆm p+1 by running a linear regression of y on x, x 2, x p+3. A starting choice for w 0 may be 1. Finally, note that there are a variety of other bandwidth selectors used in practice (e.g., cross-validation or AIC c [Hurvich et. al (1998)] ). Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
18 η = True Curve 0.4 True Curve η = η = Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
19 Generalizing to Multiple Regression Consider, for illustration, the case of a bivariate nonparametric regression problem: As before, we can take a first-order approximation of the regression function: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
20 Subbing this back into our objective function produces: We can then formulate a weighted least-squares type objective function, as before: where K is a 2 dimesnional kernel, H is the bandwidth or smoothing matrix, and Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
21 Partially Linear Models Consider a model of the form: This model is often called a semilinear or partially linear model. Here, we assume that the z s (which can be large in number) enter in a linear fashion, and the x, still assumed a scalar, enters nonparametrically. Two questions naturally arise: How should we estimate m? What about β? Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
22 Partially Linear Models Robinson s (1998 Econometrica) Estimator: Given the above specification, note: This implies: If we knew each of the conditional mean functions, we could just run a least squares regression. Robinson s idea is to estimate each of these CMF s nonparametrically, as we have discussed. Thus, we can estimate β by running least squares using the following regression: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
23 Partially Linear Models 1 Since β converges at the standard parametric rate (we can show this), we can ignore the fact that it is estimated (asymptotically) when deriving confidence intervals for m. (Some focus on β as the parameter of interest). 2 This procedure can be quite computationally intensive, since we need to perform k z + 2 nonparametric regressions in total, where k z is the number of variables in Z. 3 This estimator is asymptotically efficient. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
24 Partially Linear Models An alternate method has been suggested by Yatchew (1997 Economics Letters). He suggests the use of differencing to eliminate the unknown function m. Note that for a continuous m with x i x j : This intuition suggests the following simple estimator: Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
25 Partially Linear Models 1 Sort the data by ascending values of X. 2 Take adjacent differences of the sorted data, and estimate β by an OLS regression of the differenced y s on the differenced z s. 3 Given ˆβ, estimate the unknown function m pointwise using local linear regression of y z ˆβ on x, or from an alternate nonparametric estimation procedure. Over a compact support, and under certain regularity conditions on m the differencing technique asymptotically purges the model of the nonparametric component m, and consistent estimates of β are obtained. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
26 Partially Linear Models Yatchew (1997, 1998) describes how higher-order optimal differencing can be applied to estimate β, and approach the efficiency of Robinson s estimator as the order of differencing gets large. Note that this estimator only requires one nonparametric regression! Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
27 Smooth Coefficient Models (Li et al (2002, JBES)) Consider the following smooth coefficient model pause y i = α(z i ) + x i β(z i ) + ɛ i = X i δ(z i ), where X i = [1 x i ], δ(z i) = [α(z i ) β(z i ) ]. We can think of β(z i ) as a vector of (smooth) coefficient that depend on z. The standard partially linear model follows as β(z i ) = β. Let z be q 1 and x be p 1. (Typically, think of both q and z being equal to 1). They suggest the following estimator: ˆδ(z 0 ) = (nh q ) 1 n j=1 X jy j K ( ) zj z 0 h (nh q ) 1 n j=1 X jx j K ( zj z 0 h ). Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
28 Smooth Coefficient Models (Li et al (2002, JBES)) Intuition: This is like a weighted least squares rule. Suppose that z is a scalar, and assume that we are using a uniform kernel: { 1/2h if zj z K(x) = 0 h 0 otherwise Under this rule, we can see that ˆδ(z 0 ) = j: z j z 0 <h X j X j 1 j: z j z 0 <h X j y j. This is the least squares estimator of the intercept and slopes, using only those data points for which z j is close to z 0. Doing this over a grid of z 0 values will enable us to piece together the intercept function (as a function of z) and the slope coefficients (also as a function of z). Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
29 Tests Against Parametric Alternatives Li et al also provide a way to test against parametric alternatives. They consider a parametric version of the model: y i = X i δ 0 (z i ) + ɛ i, with δ 0 (z i ) being a particular parametric function of z, for example: y i = α 0 + γ 0 z i + x i β + ɛ i imples that X i = [1 x i ], δ 0 (z) = [(α 0 + z i γ 0 ) β 0 ]. We would like to test H 0 : H A : δ(z) δ 0 (z) a.e. = 0 δ(z) δ 0 (z) 0 on a set with positive measure Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
30 Tests Against Parametric Alternatives They propose the test statistic: Î n = (n 2 h q ) 1 i j i X i X jˆɛ i ˆɛ j K ( zi z j h ), where They also show where ˆɛ i = y i X i ˆδ0 (z i ). J n = nhq/2 Î n ˆσ 0 N(0, 1), ˆσ 2 0 = 2(n 2 h q ) 1 i j i ( ) ˆɛ 2 i ˆɛ 2 j K 2 zi z j. h Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
31 Tests Against Parametric Alternatives Notes: A rule of thumb for the bandwidth choice is h l = z l,sd n 1/(4+q), where z l,sd is the sample standard deviation of z l. Justin L. Tobias (Purdue) Nonparametric Regression April 8, / 31
Density estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas
0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity
More informationNonparametric Density Estimation
Nonparametric Density Estimation Econ 690 Purdue University Justin L. Tobias (Purdue) Nonparametric Density Estimation 1 / 29 Density Estimation Suppose that you had some data, say on wages, and you wanted
More informationNonparametric Econometrics
Applied Microeconometrics with Stata Nonparametric Econometrics Spring Term 2011 1 / 37 Contents Introduction The histogram estimator The kernel density estimator Nonparametric regression estimators Semi-
More informationRegression #2. Econ 671. Purdue University. Justin L. Tobias (Purdue) Regression #2 1 / 24
Regression #2 Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #2 1 / 24 Estimation In this lecture, we address estimation of the linear regression model. There are many objective functions
More informationNew Local Estimation Procedure for Nonparametric Regression Function of Longitudinal Data
ew Local Estimation Procedure for onparametric Regression Function of Longitudinal Data Weixin Yao and Runze Li The Pennsylvania State University Technical Report Series #0-03 College of Health and Human
More informationRegression #3: Properties of OLS Estimator
Regression #3: Properties of OLS Estimator Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #3 1 / 20 Introduction In this lecture, we establish some desirable properties associated with
More informationNonparametric Regression Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction
Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Univariate Kernel Regression The relationship between two variables, X and Y where m(
More informationLocal linear multiple regression with variable. bandwidth in the presence of heteroscedasticity
Local linear multiple regression with variable bandwidth in the presence of heteroscedasticity Azhong Ye 1 Rob J Hyndman 2 Zinai Li 3 23 January 2006 Abstract: We present local linear estimator with variable
More informationNonparametric Methods
Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis
More informationGLS and FGLS. Econ 671. Purdue University. Justin L. Tobias (Purdue) GLS and FGLS 1 / 22
GLS and FGLS Econ 671 Purdue University Justin L. Tobias (Purdue) GLS and FGLS 1 / 22 In this lecture we continue to discuss properties associated with the GLS estimator. In addition we discuss the practical
More informationRegression #4: Properties of OLS Estimator (Part 2)
Regression #4: Properties of OLS Estimator (Part 2) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #4 1 / 24 Introduction In this lecture, we continue investigating properties associated
More informationNonparametric Regression. Changliang Zou
Nonparametric Regression Institute of Statistics, Nankai University Email: nk.chlzou@gmail.com Smoothing parameter selection An overall measure of how well m h (x) performs in estimating m(x) over x (0,
More informationTime Series and Forecasting Lecture 4 NonLinear Time Series
Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations
More informationTest for Discontinuities in Nonparametric Regression
Communications of the Korean Statistical Society Vol. 15, No. 5, 2008, pp. 709 717 Test for Discontinuities in Nonparametric Regression Dongryeon Park 1) Abstract The difference of two one-sided kernel
More informationModel-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego
Model-free prediction intervals for regression and autoregression Dimitris N. Politis University of California, San Diego To explain or to predict? Models are indispensable for exploring/utilizing relationships
More informationFunction of Longitudinal Data
New Local Estimation Procedure for Nonparametric Regression Function of Longitudinal Data Weixin Yao and Runze Li Abstract This paper develops a new estimation of nonparametric regression functions for
More informationBickel Rosenblatt test
University of Latvia 28.05.2011. A classical Let X 1,..., X n be i.i.d. random variables with a continuous probability density function f. Consider a simple hypothesis H 0 : f = f 0 with a significance
More informationLocal regression I. Patrick Breheny. November 1. Kernel weighted averages Local linear regression
Local regression I Patrick Breheny November 1 Patrick Breheny STA 621: Nonparametric Statistics 1/27 Simple local models Kernel weighted averages The Nadaraya-Watson estimator Expected loss and prediction
More informationIntroduction. Linear Regression. coefficient estimates for the wage equation: E(Y X) = X 1 β X d β d = X β
Introduction - Introduction -2 Introduction Linear Regression E(Y X) = X β +...+X d β d = X β Example: Wage equation Y = log wages, X = schooling (measured in years), labor market experience (measured
More informationModel Specification Testing in Nonparametric and Semiparametric Time Series Econometrics. Jiti Gao
Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics Jiti Gao Department of Statistics School of Mathematics and Statistics The University of Western Australia Crawley
More informationECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd
ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation Petra E. Todd Fall, 2014 2 Contents 1 Review of Stochastic Order Symbols 1 2 Nonparametric Density Estimation 3 2.1 Histogram
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More informationA nonparametric method of multi-step ahead forecasting in diffusion processes
A nonparametric method of multi-step ahead forecasting in diffusion processes Mariko Yamamura a, Isao Shoji b a School of Pharmacy, Kitasato University, Minato-ku, Tokyo, 108-8641, Japan. b Graduate School
More informationTransformation and Smoothing in Sample Survey Data
Scandinavian Journal of Statistics, Vol. 37: 496 513, 2010 doi: 10.1111/j.1467-9469.2010.00691.x Published by Blackwell Publishing Ltd. Transformation and Smoothing in Sample Survey Data YANYUAN MA Department
More informationPenalized Splines, Mixed Models, and Recent Large-Sample Results
Penalized Splines, Mixed Models, and Recent Large-Sample Results David Ruppert Operations Research & Information Engineering, Cornell University Feb 4, 2011 Collaborators Matt Wand, University of Wollongong
More informationRegression #5: Confidence Intervals and Hypothesis Testing (Part 1)
Regression #5: Confidence Intervals and Hypothesis Testing (Part 1) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #5 1 / 24 Introduction What is a confidence interval? To fix ideas, suppose
More informationNonparametric Inference via Bootstrapping the Debiased Estimator
Nonparametric Inference via Bootstrapping the Debiased Estimator Yen-Chi Chen Department of Statistics, University of Washington ICSA-Canada Chapter Symposium 2017 1 / 21 Problem Setup Let X 1,, X n be
More informationIntroduction to Nonparametric Regression
Introduction to Nonparametric Regression Nathaniel E. Helwig Assistant Professor of Psychology and Statistics University of Minnesota (Twin Cities) Updated 04-Jan-2017 Nathaniel E. Helwig (U of Minnesota)
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationGeographically Weighted Regression as a Statistical Model
Geographically Weighted Regression as a Statistical Model Chris Brunsdon Stewart Fotheringham Martin Charlton October 6, 2000 Spatial Analysis Research Group Department of Geography University of Newcastle-upon-Tyne
More informationAn introduction to nonparametric and semi-parametric econometric methods
An introduction to nonparametric and semi-parametric econometric methods Robert Breunig Australian National University robert.breunig@anu.edu.au http://econrsss.anu.edu.au/staff/breunig/course_bb.htm March
More informationECO Class 6 Nonparametric Econometrics
ECO 523 - Class 6 Nonparametric Econometrics Carolina Caetano Contents 1 Nonparametric instrumental variable regression 1 2 Nonparametric Estimation of Average Treatment Effects 3 2.1 Asymptotic results................................
More informationDESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA
Statistica Sinica 18(2008), 515-534 DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA Kani Chen 1, Jianqing Fan 2 and Zhezhen Jin 3 1 Hong Kong University of Science and Technology,
More informationHypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33
Hypothesis Testing Econ 690 Purdue University Justin L. Tobias (Purdue) Testing 1 / 33 Outline 1 Basic Testing Framework 2 Testing with HPD intervals 3 Example 4 Savage Dickey Density Ratio 5 Bartlett
More informationSome Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model
Some Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model 1. Introduction Varying-coefficient partially linear model (Zhang, Lee, and Song, 2002; Xia, Zhang, and Tong, 2004;
More informationEconomics 620, Lecture 19: Introduction to Nonparametric and Semiparametric Estimation
Economics 620, Lecture 19: Introduction to Nonparametric and Semiparametric Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 19: Nonparametric Analysis
More informationNonparametric Modal Regression
Nonparametric Modal Regression Summary In this article, we propose a new nonparametric modal regression model, which aims to estimate the mode of the conditional density of Y given predictors X. The nonparametric
More information4 Nonparametric Regression
4 Nonparametric Regression 4.1 Univariate Kernel Regression An important question in many fields of science is the relation between two variables, say X and Y. Regression analysis is concerned with the
More informationLocal Polynomial Modelling and Its Applications
Local Polynomial Modelling and Its Applications J. Fan Department of Statistics University of North Carolina Chapel Hill, USA and I. Gijbels Institute of Statistics Catholic University oflouvain Louvain-la-Neuve,
More informationF9 F10: Autocorrelation
F9 F10: Autocorrelation Feng Li Department of Statistics, Stockholm University Introduction In the classic regression model we assume cov(u i, u j x i, x k ) = E(u i, u j ) = 0 What if we break the assumption?
More informationDiscussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon
Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics
More informationModelling Non-linear and Non-stationary Time Series
Modelling Non-linear and Non-stationary Time Series Chapter 2: Non-parametric methods Henrik Madsen Advanced Time Series Analysis September 206 Henrik Madsen (02427 Adv. TS Analysis) Lecture Notes September
More informationAlternatives to Basis Expansions. Kernels in Density Estimation. Kernels and Bandwidth. Idea Behind Kernel Methods
Alternatives to Basis Expansions Basis expansions require either choice of a discrete set of basis or choice of smoothing penalty and smoothing parameter Both of which impose prior beliefs on data. Alternatives
More informationLocal linear multivariate. regression with variable. bandwidth in the presence of. heteroscedasticity
Model ISSN 1440-771X Department of Econometrics and Business Statistics http://www.buseco.monash.edu.au/depts/ebs/pubs/wpapers/ Local linear multivariate regression with variable bandwidth in the presence
More informationSection 7: Local linear regression (loess) and regression discontinuity designs
Section 7: Local linear regression (loess) and regression discontinuity designs Yotam Shem-Tov Fall 2015 Yotam Shem-Tov STAT 239/ PS 236A October 26, 2015 1 / 57 Motivation We will focus on local linear
More informationLocal Polynomial Estimation for Sensitivity Analysis on Models With Correlated Inputs
Local Polynomial Estimation for Sensitivity Analysis on Models With Correlated Inputs Sébastien Da Veiga, François Wahl, Fabrice Gamboa To cite this version: Sébastien Da Veiga, François Wahl, Fabrice
More informationLOCAL POLYNOMIAL REGRESSION ON UNKNOWN MANIFOLDS. Department of Statistics. University of California at Berkeley, USA. 1.
LOCAL POLYNOMIAL REGRESSION ON UNKNOWN MANIFOLDS PETER J. BICKEL AND BO LI Department of Statistics University of California at Berkeley, USA Abstract. We reveal the phenomenon that naive multivariate
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationGibbs Sampling in Linear Models #1
Gibbs Sampling in Linear Models #1 Econ 690 Purdue University Justin L Tobias Gibbs Sampling #1 Outline 1 Conditional Posterior Distributions for Regression Parameters in the Linear Model [Lindley and
More informationPREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY
REVSTAT Statistical Journal Volume 7, Number 1, April 2009, 37 54 PREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY Authors: Germán Aneiros-Pérez Departamento de Matemáticas,
More informationRegression #8: Loose Ends
Regression #8: Loose Ends Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #8 1 / 30 In this lecture we investigate a variety of topics that you are probably familiar with, but need to touch
More informationRegression Discontinuity Designs
Regression Discontinuity Designs Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 1 / 1 Observational
More informationSimple and Efficient Improvements of Multivariate Local Linear Regression
Journal of Multivariate Analysis Simple and Efficient Improvements of Multivariate Local Linear Regression Ming-Yen Cheng 1 and Liang Peng Abstract This paper studies improvements of multivariate local
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationStatistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation
Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence
More informationDISCUSSION PAPER. The Bias from Misspecification of Control Variables as Linear. L e o n a r d G o f f. November 2014 RFF DP 14-41
DISCUSSION PAPER November 014 RFF DP 14-41 The Bias from Misspecification of Control Variables as Linear L e o n a r d G o f f 1616 P St. NW Washington, DC 0036 0-38-5000 www.rff.org The Bias from Misspecification
More informationNonparametric Regression. Badr Missaoui
Badr Missaoui Outline Kernel and local polynomial regression. Penalized regression. We are given n pairs of observations (X 1, Y 1 ),...,(X n, Y n ) where Y i = r(x i ) + ε i, i = 1,..., n and r(x) = E(Y
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationNonparametric Estimation of the Marginal Effect in Fixed-Effect Panel Data Models
Nonparametric Estimation of the Marginal Effect in Fixed-Effect Panel Data Models Yoonseok Lee Debasri Mukherjee Aman Ullah October 207 Abstract This paper considers local linear least squares estimation
More informationEconometrics. Week 11. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague
Econometrics Week 11 Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Fall 2012 1 / 30 Recommended Reading For the today Advanced Time Series Topics Selected topics
More informationLecture 02 Linear classification methods I
Lecture 02 Linear classification methods I 22 January 2016 Taylor B. Arnold Yale Statistics STAT 365/665 1/32 Coursewebsite: A copy of the whole course syllabus, including a more detailed description of
More informationLinear IV and Simultaneous Equations
Linear IV and Daniel Schmierer Econ 312 April 6, 2007 Setup Linear regression model Y = X β + ε (1) Endogeneity of X means that X and ε are correlated, ie. E(X ε) 0. Suppose we observe another variable
More informationStructural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall
1 Structural Nested Mean Models for Assessing Time-Varying Effect Moderation Daniel Almirall Center for Health Services Research, Durham VAMC & Dept. of Biostatistics, Duke University Medical Joint work
More informationOn the Robust Modal Local Polynomial Regression
International Journal of Statistical Sciences ISSN 683 5603 Vol. 9(Special Issue), 2009, pp 27-23 c 2009 Dept. of Statistics, Univ. of Rajshahi, Bangladesh On the Robust Modal Local Polynomial Regression
More informationConfidence intervals for kernel density estimation
Stata User Group - 9th UK meeting - 19/20 May 2003 Confidence intervals for kernel density estimation Carlo Fiorio c.fiorio@lse.ac.uk London School of Economics and STICERD Stata User Group - 9th UK meeting
More informationKernel density estimation
Kernel density estimation Patrick Breheny October 18 Patrick Breheny STA 621: Nonparametric Statistics 1/34 Introduction Kernel Density Estimation We ve looked at one method for estimating density: histograms
More informationLOCAL LINEAR REGRESSION FOR GENERALIZED LINEAR MODELS WITH MISSING DATA
The Annals of Statistics 1998, Vol. 26, No. 3, 1028 1050 LOCAL LINEAR REGRESSION FOR GENERALIZED LINEAR MODELS WITH MISSING DATA By C. Y. Wang, 1 Suojin Wang, 2 Roberto G. Gutierrez and R. J. Carroll 3
More informationNonparametric Estimation of Regression Functions In the Presence of Irrelevant Regressors
Nonparametric Estimation of Regression Functions In the Presence of Irrelevant Regressors Peter Hall, Qi Li, Jeff Racine 1 Introduction Nonparametric techniques robust to functional form specification.
More informationStructural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall
1 Structural Nested Mean Models for Assessing Time-Varying Effect Moderation Daniel Almirall Center for Health Services Research, Durham VAMC & Duke University Medical, Dept. of Biostatistics Joint work
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationThe Simple Linear Regression Model
The Simple Linear Regression Model Lesson 3 Ryan Safner 1 1 Department of Economics Hood College ECON 480 - Econometrics Fall 2017 Ryan Safner (Hood College) ECON 480 - Lesson 3 Fall 2017 1 / 77 Bivariate
More informationError distribution function for parametrically truncated and censored data
Error distribution function for parametrically truncated and censored data Géraldine LAURENT Jointly with Cédric HEUCHENNE QuantOM, HEC-ULg Management School - University of Liège Friday, 14 September
More informationSection Properties of Rational Expressions
88 Section. - Properties of Rational Expressions Recall that a rational number is any number that can be written as the ratio of two integers where the integer in the denominator cannot be. Rational Numbers:
More informationA New Method for Varying Adaptive Bandwidth Selection
IEEE TRASACTIOS O SIGAL PROCESSIG, VOL. 47, O. 9, SEPTEMBER 1999 2567 TABLE I SQUARE ROOT MEA SQUARED ERRORS (SRMSE) OF ESTIMATIO USIG THE LPA AD VARIOUS WAVELET METHODS A ew Method for Varying Adaptive
More informationNADARAYA WATSON ESTIMATE JAN 10, 2006: version 2. Y ik ( x i
NADARAYA WATSON ESTIMATE JAN 0, 2006: version 2 DATA: (x i, Y i, i =,..., n. ESTIMATE E(Y x = m(x by n i= ˆm (x = Y ik ( x i x n i= K ( x i x EXAMPLES OF K: K(u = I{ u c} (uniform or box kernel K(u = u
More informationDay 4A Nonparametrics
Day 4A Nonparametrics A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Economics Norwegian School of Economics Advanced Microeconometrics Aug 28 - Sep 2, 2017. Colin Cameron Univ. of Calif.
More informationChapter 2: simple regression model
Chapter 2: simple regression model Goal: understand how to estimate and more importantly interpret the simple regression Reading: chapter 2 of the textbook Advice: this chapter is foundation of econometrics.
More informationRegression I: Mean Squared Error and Measuring Quality of Fit
Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving
More informationWEIGHTED QUANTILE REGRESSION THEORY AND ITS APPLICATION. Abstract
Journal of Data Science,17(1). P. 145-160,2019 DOI:10.6339/JDS.201901_17(1).0007 WEIGHTED QUANTILE REGRESSION THEORY AND ITS APPLICATION Wei Xiong *, Maozai Tian 2 1 School of Statistics, University of
More informationOn a Nonparametric Notion of Residual and its Applications
On a Nonparametric Notion of Residual and its Applications Bodhisattva Sen and Gábor Székely arxiv:1409.3886v1 [stat.me] 12 Sep 2014 Columbia University and National Science Foundation September 16, 2014
More informationWeb-based Supplementary Material for. Dependence Calibration in Conditional Copulas: A Nonparametric Approach
1 Web-based Supplementary Material for Dependence Calibration in Conditional Copulas: A Nonparametric Approach Elif F. Acar, Radu V. Craiu, and Fang Yao Web Appendix A: Technical Details The score and
More informationNonparametric Small Area Estimation Using Penalized Spline Regression
Nonparametric Small Area Estimation Using Penalized Spline Regression 0verview Spline-based nonparametric regression Nonparametric small area estimation Prediction mean squared error Bootstrapping small
More informationTitolo Smooth Backfitting with R
Rapporto n. 176 Titolo Smooth Backfitting with R Autori Alberto Arcagni, Luca Bagnato ottobre 2009 Dipartimento di Metodi Quantitativi per le Scienze Economiche ed Aziendali Università degli Studi di Milano
More information3 Nonparametric Density Estimation
3 Nonparametric Density Estimation Example: Income distribution Source: U.K. Family Expenditure Survey (FES) 1968-1995 Approximately 7000 British Households per year For each household many different variables
More informationSINGLE-STEP ESTIMATION OF A PARTIALLY LINEAR MODEL
SINGLE-STEP ESTIMATION OF A PARTIALLY LINEAR MODEL DANIEL J. HENDERSON AND CHRISTOPHER F. PARMETER Abstract. In this paper we propose an asymptotically equivalent single-step alternative to the two-step
More informationA Design Unbiased Variance Estimator of the Systematic Sample Means
American Journal of Theoretical and Applied Statistics 2015; 4(3): 201-210 Published online May 29, 2015 (http://www.sciencepublishinggroup.com/j/ajtas) doi: 10.1148/j.ajtas.20150403.27 ISSN: 232-8999
More informationSEMIPARAMETRIC APPLICATIONS IN ECONOMIC GROWTH. Mustafa Koroglu. A Thesis presented to The University of Guelph
SEMIPARAMETRIC APPLICATIONS IN ECONOMIC GROWTH by Mustafa Koroglu A Thesis presented to The University of Guelph In partial fulfilment of requirements for the degree of Doctor of Philosophy in Economics
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationA COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES
A COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES MICHAEL O HARA AND CHRISTOPHER F. PARMETER Abstract. This paper presents a Monte Carlo comparison of
More informationConvergence rates for uniform confidence intervals based on local polynomial regression estimators
Journal of Nonparametric Statistics ISSN: 1048-5252 Print) 1029-0311 Online) Journal homepage: http://www.tandfonline.com/loi/gnst20 Convergence rates for uniform confidence intervals based on local polynomial
More information4. Nonlinear regression functions
4. Nonlinear regression functions Up to now: Population regression function was assumed to be linear The slope(s) of the population regression function is (are) constant The effect on Y of a unit-change
More informationSolving a Series. Carmen Bruni
A Sample Series Problem Question: Does the following series converge or diverge? n=1 n 3 + 3n 2 + 1 n 5 + 14n 3 + 4n First Attempt First let s think about what this series is - maybe the terms are big
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationOn Fractile Transformation of Covariates in Regression 1
1 / 13 On Fractile Transformation of Covariates in Regression 1 Bodhisattva Sen Department of Statistics Columbia University, New York ERCIM 10 11 December, 2010 1 Joint work with Probal Chaudhuri, Indian
More informationDepartment of Statistics Purdue University West Lafayette, IN USA
Effect of Mean on Variance Funtion Estimation in Nonparametric Regression by Lie Wang, Lawrence D. Brown,T. Tony Cai University of Pennsylvania Michael Levine Purdue University Technical Report #06-08
More informationWorking Paper No Maximum score type estimators
Warsaw School of Economics Institute of Econometrics Department of Applied Econometrics Department of Applied Econometrics Working Papers Warsaw School of Economics Al. iepodleglosci 64 02-554 Warszawa,
More informationGeneralized Additive Models
Generalized Additive Models The Model The GLM is: g( µ) = ß 0 + ß 1 x 1 + ß 2 x 2 +... + ß k x k The generalization to the GAM is: g(µ) = ß 0 + f 1 (x 1 ) + f 2 (x 2 ) +... + f k (x k ) where the functions
More informationNonparametric Econometrics in R
Nonparametric Econometrics in R Philip Shaw Fordham University November 17, 2011 Philip Shaw (Fordham University) Nonparametric Econometrics in R November 17, 2011 1 / 16 Introduction The NP Package R
More informationProblem Set 2 Solution Sketches Time Series Analysis Spring 2010
Problem Set 2 Solution Sketches Time Series Analysis Spring 2010 Forecasting 1. Let X and Y be two random variables such that E(X 2 ) < and E(Y 2 )
More informationEstimation of Treatment Effects under Essential Heterogeneity
Estimation of Treatment Effects under Essential Heterogeneity James Heckman University of Chicago and American Bar Foundation Sergio Urzua University of Chicago Edward Vytlacil Columbia University March
More information