Introduction. Linear Regression. coefficient estimates for the wage equation: E(Y X) = X 1 β X d β d = X β
|
|
- Osborn Carson
- 5 years ago
- Views:
Transcription
1 Introduction - Introduction -2 Introduction Linear Regression E(Y X) = X β +...+X d β d = X β Example: Wage equation Y = log wages, X = schooling (measured in years), labor market experience (measured as: AGE SCHOOL 6) and experience squared. E(Y SCHOOL, EXP) = β +β 2 SCHOOL+β 3 EXP+β 4 EXP 2. coefficient estimates for the wage equation: Dependent Variable: Log Wages Variable Coefficients S.E. t-values SCHOOL EXP EXP constant R 2 = 0.24, sample size n = 534 Table : Results from OLS estimation CPS 985, n = 534 Berndt (99) Introduction -3 Introduction -4 Wage <-- Schooling Wage <-- Experience Wage <-- Schooling, Experience Experience Schooling 4.0 Figure : wage-schooling profile and wage-experience profile R: SPMcps85lin Figure 2: Parametrically estimated regression function R: SPMcps85lin
2 Introduction -5 Introduction -6 Wage <-- Schooling, Experience Linear regression E(Y SCHOOL,EXP) = const+β SCHOOL+β 2 EXP 2.4 Nonparametric Regression 2.0 m(.) is a smooth function E(Y SCHOOL, EXP) = m(school, EXP) 4.2 Experience Schooling 4.0 Figure 3: Nonparametrically estimated regression function R: SPMcps85reg Introduction -7 Introduction -8 Engel Curve Example: Engel curve Engel (857)..., daß je ärmer eine Familie ist, einen desto größeren Anteil von der Gesammtausgabe muß zur Beschaffung der Nahrung aufgewendet werden. (The poorer a family, the bigger the share of total expenditures that has to be used for food.) Figure 4: Engel curve, U.K. Family Expenditure Survey 973 R: SPMengelcurve2
3 Introduction - 3 Introduction - 4 Example: Wage equation Nonparametric regression E(Y SCHOOL, EXP) = m(school, EXP) Semiparametric Regression E(Y SCHOOL,EXP) = α+g (SCHOOL)+g 2 (EXP) Y Wage <-- Schooling Y Wage <-- Experience g (.), g 2 (.) are smooth functions X X Figure 8: Additive model fit vs. parametric fit, wage-schooling (left) and wage-experience (right) profiles R: SPMcps85add Introduction - 5 Introduction - 6 Wage <-- Schooling, Experience Example: Binary choice model if person imagines to move to west, Y = 0 otherwise E(Y x) = P(Y = x) = G(β x) 4.2 Experience Schooling 4.0 typically: logistic link function (logit model) E(Y x) = P(Y = x) = +exp( β x) Figure 9: Surface plot for the additive model R: SPMcps85add
4 Introduction - 7 Introduction - 8 Logit Model Link Function, Responses Heteroscedasticity problems binary choice model with Var(ε) = 4 where u has a (standard) logistic distribution { +(β x) 2} 2 Var(u), Index Figure 0: Logit model for migration R: SPMlogit Introduction - 9 Introduction - 20 Wrong link consequences True versus Logit Link G(Index) Index Sampling Distribution True Ratio vs. Sampling Distribution True+Estimated Ratio Figure : The link function of a homoscedastic logit model (thin) vs. a heteroscedastic model (solid) R: SPMtruelogit Figure 2: Sampling distribution of the ratio of estimated coefficients and the ratio s true value R: SPMsimulogit
5 Introduction - 2 Introduction - 22 Link Function, Responses Single Index Model Index Summary: Introduction Parametric models are fully determined up to a parameter (vector). They lead to an easy interpretation of the resulting fits. Nonparametric models only impose qualitative restrictions like a smooth density function or a smooth regression function m. They may support known parametric models. They open the way for new models by their flexibility. Semiparametric model combine parametric and nonparametric parts. This keeps the easy interpretation of the results, but gives more flexibility in some aspects of the model. Figure 3: Single index model for migration R: SPMsim Nonparametric Regression 4- Nonparametric Regression {(X i,y i )}, i =,...,n; X R d,y R Engel curve: X = net-income, Y = expenditure Y = m(x)+ε CHARN model: time series of the form Y t = m(y t )+σ(y t )ξ t Nonparametric Regression 4-2 Univariate Kernel Regression model Y i = m(x i )+ε i, i =,...,n m( ) smooth regression function, ε i i.i.d. error terms with Eε i = 0 we aim to estimate the conditional expectation of Y given X = x m(x) = E(Y X = x) = y f(y x) dy = y f(x,y) f X (x) dy where f(x,y) denotes the joint density of (X,Y) and f X (x) the marginal density of X Example: normal variables (X,Y) N(µ,Σ) = m(x) is linear.
6 Nonparametric Regression 4-4 Nonparametric Regression 4-5 Nadaraya-Watson Estimator idea: (X i,y i ) have joint a pdf, so we can estimate m( ) by a multivariate kernel estimator f h, h(x,y) = n ( ) ( ) n h K x Xi y Yi h hk h resulting estimator = y f h, h(x,y)dy = n n K h (x X i )Y i n n K h (x X i )Y i m h (x) = n n K h (x X j ) j= = r h(x) f h (x) Engel Curve Figure 44: Nadaraya-Watson kernel regression, h = 0.2, U.K. Family Expenditure Survey 973 R: SPMengelcurve Nonparametric Regression 4-8 Nonparametric Regression 4-9 Statistical properties of the Nadaraya-Watson estimator m h (x) m(x) = = r h(x) m(x) f h (x) f X (x) { }[ { r h (x) fh f h (x) m(x) (x) f X (x) + f }] h (x) f X (x) +{ m h (x) m(x)} f X(x) f h (x) f X (x) calculate now bias and variance in the same way as for the KDE: AMSE{ m h (x)} = σ 2 (x) nh f X (x) K 2 2 }{{} variance part { + h4 m (x)+2 m (x)f X (x) 4 f X (x) } 2 µ 2 2(K) } {{ } bias part h=0.05 h=0. h=0.2 h=0.5 Figure 45: Four kernel regression estimates for the 973 U.K. Family Expenditure data with bandwidths h = 0.05, h = 0., h = 0.2, and h = 0.5 R: SPMregress
7 Nonparametric Regression 4-0 Nonparametric Regression 4- Asymptotic normal distribution For some regularity conditions and h = cn /5 remarks n 2/5 { m h (x) m(x)} L N { m (x) c2 µ 2 (K) + m (x)f X (x) } 2 f X (x) }{{} b x bias is a function of m and m f variance is a function of σ 2 and f, σ2 (x) K 2 2 cf X (x) } {{ } vx 2. Pointwise Confidence Intervals [ m h (x) z α 2 K 2 σ 2 (x) nh f h (x), m h (x)+z α 2 K 2 σ 2 (x) n σ 2 (x) = n W hi (x){y i m h (x)} 2, nh f h (x) σ 2 and f both influence the precision of the confidence interval correction for bias!? analogous to KDE: confidence bands (Bickel and Rosenblatt; 973) ] Nonparametric Regression 4-2 Nonparametric Regression 4-3 Confidence Intervals Local Polynomial Estimation Taylor expansion for sufficiently smooth functions m(t) m(x)+m (x)(t x)+m (x)(t x) 2 2! + +m(p) (x)(t x) p p! consider a weighted least squares problem min β n { Y i β 0 β (X i x) β 2 (X i x) 2... β p (X i x) p} 2 Kh (x X i ) Figure 46: Nadaraya-Watson kernel regression and 95% confidence intervals, h = 0.2, U.K. Family Expenditure Survey 973 R: SPMengelconf = resulting estimate for β provides estimates for m (ν) (x), ν = 0,,...,p
8 Nonparametric Regression 4-4 Nonparametric Regression 4-5 notations X x (X x) 2... (X x) p X 2 x (X 2 x) 2... (X 2 x) p X = X n x (X n x) 2... (X n x) p Y = (Y,,Y n ) W = diag({k h (x X i )} n ) local polynomial estimator β(x) = ( X WX ) X WY local polynomial regression estimator m h,p (x) = β 0 (x) (Asymptotic) Statistical Properties under regularity conditions, h = cn 5, nh remarks: n 2/5 { m h, (x) m(x)} L N asymptotically equivalent to higher order kernel ( c 2 µ 2 (K) m (x), σ2 (x) K cf X (x) analog theorem can be stated for derivative estimation ) Nonparametric Regression 4-6 Nonparametric Regression 4-7 Local Polynomial Regression Derivative Estimation Figure 47: Local polynomial regression, p =, h = 0.2, U.K. Family Expenditure Survey 973 R: SPMlocpolyreg Figure 48: Local polynomial derivative estimation, p = 2, h by rule of thumb, U.K. Family Expenditure Survey 973 R: SPMderivest
9 Kernel Density Estimation 3-3 Kernel Density Estimation 3-4 Different Kernel Functions Required properties of kernels K( ) is a density function: K( ) is symmetric: K(u)du = and K(u) 0 uk(u)du = 0 Kernel K(u) Uniform 2 ( u ) Triangle ( u )( u ) 3 Epanechnikov 4 ( u2 )( u ) 5 Quartic 6 ( u2 ) 2 ( u ) 35 Triweight 32 ( u2 ) 3 ( u ) Gaussian 2π exp( 2 u2 ) π Cosinus 4 cos(π 2u)( u ) Table 2: Kernel functions Kernel Density Estimation 3-5 Kernel Density Estimation 3-6 Uniform Epanechnikov K(u) K(u) u Triangle u K(u) K(u) u Quartic Figure 27: Some kernel functions: Uniform (top left), Triangle (bottom left), Epanechnikov (top right), Quartic (bottom right) R: SPMkernel u Example: Construction of the KDE consider the KDE using a Gaussian kernel f h (x) = n ( ) x Xi K nh h here we have = nh n u = x X i h 2π exp( 2 u2 )
10 Kernel Density Estimation 3-52 Kernel Density Estimation 3-53 Example: bandwidth matrix H = 0 0 How to get a Multivariate Kernel? u = (u,...,u d ) product kernel radially symmetric or spherical kernel K(u) = K(u )... K(u d ) K( u ) K(u) = K( u )du R d with u def = u u Product Kernel Radial-symmetric Kernel Figure 37: Contours from bivariate product (left) and bivariate radially symmetric (right) Epanechnikov kernel R: SPMkernelcontours Kernel Density Estimation 3-54 Example: bandwidth matrix H = Kernel Density Estimation 3-55 Example: bandwidth matrix H = /2 Product Kernel Radial-symmetric Kernel Product Kernel Radial-symmetric Kernel Figure 38: Contours from bivariate product (left) and bivariate radially symmetric (right) Epanechnikov kernel R: SPMkernelcontours Figure 39: Contours from bivariate product (left) and bivariate radially symmetric (right) Epanechnikov kernel R: SPMkernelcontours
11 Kernel Density Estimation 3-56 Kernel properties K is a density function K is symmetric K has a second moment (matrix) R d K(u)du =, K(u) 0 R d uk(u)du = 0 d R d uu K(u)du = µ 2 (K)I d where I d denotes the d d identity matrix K has a kernel norm K 2 2 = K 2 (u)du Nonparametric Regression 4-8 Higher Order Kernels Kernel is of order p if u j K(u) du = 0 and u p K(u) du < j =,...,p K_opt(v=0,p=4,6) K_opt(v=,2,p=5,6) Figure 49: Optimal p th order kernels for v th derivative est. R: SPMhokernel Nonparametric Regression 4-9 Nonparametric Regression 4-20 Optimal and Gauss-type Higher Order Kernels Order(v, p) K(u) Opt (0,4) 5 32 (3 0u2 +7u 4 ) Opt (0,6) (5 05u2 +89u 4 99u 6 ) Opt (,3) 5 4 (u3 u) Opt (, 5) ( 5u+4u3 9u 5 ) Opt (2, 4) 05 6 ( +6u2 5u 4 ) Opt (2, 6) ( 5+63u2 35u 4 +77u 6 ) Gauss (0, 4) 2 (3 u2 )φ(u) Gauss (0, 6) (5 0u2 +u 4 )φ(u) Gauss (0, 8) 48 (05 05u2 +2u 4 u 6 )φ(u) Gauss (0, 0) 384 ( u2 +378u 4 36u 6 +u 8 )φ(u) Estimators using Higher Order Kernels Kernel estimators with higher order (p > 2) kernels achieve better bias rates (typically: K of order p bias h p ) Therefore, the asymptotic statistical properties are equivalent to those of local polynomials of corresponding order Like for local polynomials, the optimal choice is to have p v > 0 even Note that this is asymptotic. In practice, performance (especially graphical one) might be poor unless samples size is huge Can also be applied to density estimation but may give negative estimates for some x-values, especially where data are sparse or samples are small
12 Nonparametric Regression 4-2 Bandwidth Choice in Kernel Regression Nonparametric Regression 4-22 Example: simulated data for previous figure Bias^2, Variance and MASE Simulated Data Bias^2, Variance and MASE y, m, mh Bandwidth h x Figure 50: MASE (thick line), bias part (thin solid line) and variance part (thin dashed line) for simulated data R: SPMsimulmase Figure 5: Simulated data with curve m(x) = {sin(2πx 3 )} 3 Y i = m(x i) + ε i,x i U[0,],ε i N(0,0.) R: SPMsimulmase Nonparametric Regression 4-23 Convergence rates (univariate case) for a kernel K of order p AMSE(h) = nh C +h 2p C 2 Nonparametric Regression 4-24 Cross Validation n ASE = n { m h (X j ) m(x j )} 2 w(x j ) j= MASE = E{ASE X = x,,x n = x n } Plug-In Methods h opt = argmin h AMISE(h) = h opt n +2p, AMISE(hopt ) n 2p 2p+ Idea: calculate pre-estimators of all unknown parts in C and C 2 Most practical: rough approximations should do, e.g. use some parametric higher order polynomials (at least of order two to obtain also the second derivatives) estimate MASE by resubstitution function (w is a weight function) p(h) = n {Y i m h (X i )} 2 w(x i ) n separate estimation and validation by using leave-one-out estimators CV(h) = n {Y i m h, i (X i )} 2 w(x i ) n minimizing gives ĥcv
13 Nonparametric Regression 4-25 Nonparametric Regression 4-26 Generalized Cross Validation Because CV does easily break down or is computationally too intensive for large data sets, Generalized Cross Validation has been proposed. Minimizes the residual sum of squares corrected for the degrees of freedom (dof). Unfortunately, there exist several definitions like e.g. nˆσ 2 n dof (=) RSS n dof, n RSS (n dof) 2 Also for the dof we find several definitions. Considering only estimators linear in Y, i.e. ˆm(X) = AY, typical proposals are trace(a), trace(aa t ), n trace(2a AA t ) Nadaraya-Watson Estimate Any of this definitions makes only sense for symmetric A. Minimizing gives ĥgcv Figure 52: Nadaraya-Watson kernel regression, cross-validated bandwidth ĥcv = 0.5, U.K. Family Expenditure Survey 973 R: SPMnadwaest Nonparametric Regression 4-27 Local Polynomial Estimate Figure 53: Local polynomial regression, p =, cross-validated bandwidth ĥcv = 0.56, U.K. Family Expenditure Survey 973 R: SPMlocpolyest
Nonparametric Regression Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction
Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Univariate Kernel Regression The relationship between two variables, X and Y where m(
More information4 Nonparametric Regression
4 Nonparametric Regression 4.1 Univariate Kernel Regression An important question in many fields of science is the relation between two variables, say X and Y. Regression analysis is concerned with the
More informationNonparametric Regression. Changliang Zou
Nonparametric Regression Institute of Statistics, Nankai University Email: nk.chlzou@gmail.com Smoothing parameter selection An overall measure of how well m h (x) performs in estimating m(x) over x (0,
More informationModelling Non-linear and Non-stationary Time Series
Modelling Non-linear and Non-stationary Time Series Chapter 2: Non-parametric methods Henrik Madsen Advanced Time Series Analysis September 206 Henrik Madsen (02427 Adv. TS Analysis) Lecture Notes September
More informationNonparametric Density Estimation (Multidimension)
Nonparametric Density Estimation (Multidimension) Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann February 19, 2007 Setup One-dimensional
More informationTime Series and Forecasting Lecture 4 NonLinear Time Series
Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations
More informationNonparametric Methods
Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis
More informationLocal Polynomial Regression
VI Local Polynomial Regression (1) Global polynomial regression We observe random pairs (X 1, Y 1 ),, (X n, Y n ) where (X 1, Y 1 ),, (X n, Y n ) iid (X, Y ). We want to estimate m(x) = E(Y X = x) based
More informationDensity estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas
0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationMinimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model.
Minimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model By Michael Levine Purdue University Technical Report #14-03 Department of
More informationNonparametric Econometrics
Applied Microeconometrics with Stata Nonparametric Econometrics Spring Term 2011 1 / 37 Contents Introduction The histogram estimator The kernel density estimator Nonparametric regression estimators Semi-
More informationIntroduction to Regression
Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric
More informationNonparametric Regression
Nonparametric Regression Econ 674 Purdue University April 8, 2009 Justin L. Tobias (Purdue) Nonparametric Regression April 8, 2009 1 / 31 Consider the univariate nonparametric regression model: where y
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More information3 Nonparametric Density Estimation
3 Nonparametric Density Estimation Example: Income distribution Source: U.K. Family Expenditure Survey (FES) 1968-1995 Approximately 7000 British Households per year For each household many different variables
More informationBandwidth selection for kernel conditional density
Bandwidth selection for kernel conditional density estimation David M Bashtannyk and Rob J Hyndman 1 10 August 1999 Abstract: We consider bandwidth selection for the kernel estimator of conditional density
More informationModel Specification Testing in Nonparametric and Semiparametric Time Series Econometrics. Jiti Gao
Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics Jiti Gao Department of Statistics School of Mathematics and Statistics The University of Western Australia Crawley
More informationCopula Regression RAHUL A. PARSA DRAKE UNIVERSITY & STUART A. KLUGMAN SOCIETY OF ACTUARIES CASUALTY ACTUARIAL SOCIETY MAY 18,2011
Copula Regression RAHUL A. PARSA DRAKE UNIVERSITY & STUART A. KLUGMAN SOCIETY OF ACTUARIES CASUALTY ACTUARIAL SOCIETY MAY 18,2011 Outline Ordinary Least Squares (OLS) Regression Generalized Linear Models
More informationAlternatives to Basis Expansions. Kernels in Density Estimation. Kernels and Bandwidth. Idea Behind Kernel Methods
Alternatives to Basis Expansions Basis expansions require either choice of a discrete set of basis or choice of smoothing penalty and smoothing parameter Both of which impose prior beliefs on data. Alternatives
More informationSingle Index Quantile Regression for Heteroscedastic Data
Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University JSM, 2015 E. Christou, M. G. Akritas (PSU) SIQR JSM, 2015
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/100 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression
More informationIntroduction to Regression
Introduction to Regression David E Jones (slides mostly by Chad M Schafer) June 1, 2016 1 / 102 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures
More informationComputational treatment of the error distribution in nonparametric regression with right-censored and selection-biased data
Computational treatment of the error distribution in nonparametric regression with right-censored and selection-biased data Géraldine Laurent 1 and Cédric Heuchenne 2 1 QuantOM, HEC-Management School of
More informationSingle Index Quantile Regression for Heteroscedastic Data
Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University SMAC, November 6, 2015 E. Christou, M. G. Akritas (PSU) SIQR
More informationECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd
ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation Petra E. Todd Fall, 2014 2 Contents 1 Review of Stochastic Order Symbols 1 2 Nonparametric Density Estimation 3 2.1 Histogram
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationNonparametric Regression. Badr Missaoui
Badr Missaoui Outline Kernel and local polynomial regression. Penalized regression. We are given n pairs of observations (X 1, Y 1 ),...,(X n, Y n ) where Y i = r(x i ) + ε i, i = 1,..., n and r(x) = E(Y
More informationSection 7: Local linear regression (loess) and regression discontinuity designs
Section 7: Local linear regression (loess) and regression discontinuity designs Yotam Shem-Tov Fall 2015 Yotam Shem-Tov STAT 239/ PS 236A October 26, 2015 1 / 57 Motivation We will focus on local linear
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression
More informationDEPARTMENT MATHEMATIK ARBEITSBEREICH MATHEMATISCHE STATISTIK UND STOCHASTISCHE PROZESSE
Estimating the error distribution in nonparametric multiple regression with applications to model testing Natalie Neumeyer & Ingrid Van Keilegom Preprint No. 2008-01 July 2008 DEPARTMENT MATHEMATIK ARBEITSBEREICH
More informationLocal linear multiple regression with variable. bandwidth in the presence of heteroscedasticity
Local linear multiple regression with variable bandwidth in the presence of heteroscedasticity Azhong Ye 1 Rob J Hyndman 2 Zinai Li 3 23 January 2006 Abstract: We present local linear estimator with variable
More informationBayesian estimation of bandwidths for a nonparametric regression model with a flexible error density
ISSN 1440-771X Australia Department of Econometrics and Business Statistics http://www.buseco.monash.edu.au/depts/ebs/pubs/wpapers/ Bayesian estimation of bandwidths for a nonparametric regression model
More informationModel-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego
Model-free prediction intervals for regression and autoregression Dimitris N. Politis University of California, San Diego To explain or to predict? Models are indispensable for exploring/utilizing relationships
More information12 - Nonparametric Density Estimation
ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6
More informationQuantitative Economics for the Evaluation of the European Policy. Dipartimento di Economia e Management
Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti 1 Davide Fiaschi 2 Angela Parenti 3 9 ottobre 2015 1 ireneb@ec.unipi.it. 2 davide.fiaschi@unipi.it.
More informationSmooth simultaneous confidence bands for cumulative distribution functions
Journal of Nonparametric Statistics, 2013 Vol. 25, No. 2, 395 407, http://dx.doi.org/10.1080/10485252.2012.759219 Smooth simultaneous confidence bands for cumulative distribution functions Jiangyan Wang
More informationTobit and Selection Models
Tobit and Selection Models Class Notes Manuel Arellano November 24, 2008 Censored Regression Illustration : Top-coding in wages Suppose Y log wages) are subject to top coding as is often the case with
More information18 Bivariate normal distribution I
8 Bivariate normal distribution I 8 Example Imagine firing arrows at a target Hopefully they will fall close to the target centre As we fire more arrows we find a high density near the centre and fewer
More informationAdvanced Econometrics I
Lecture Notes Autumn 2010 Dr. Getinet Haile, University of Mannheim 1. Introduction Introduction & CLRM, Autumn Term 2010 1 What is econometrics? Econometrics = economic statistics economic theory mathematics
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationLecture Notes 15 Prediction Chapters 13, 22, 20.4.
Lecture Notes 15 Prediction Chapters 13, 22, 20.4. 1 Introduction Prediction is covered in detail in 36-707, 36-701, 36-715, 10/36-702. Here, we will just give an introduction. We observe training data
More informationDay 4A Nonparametrics
Day 4A Nonparametrics A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Economics Norwegian School of Economics Advanced Microeconometrics Aug 28 - Sep 2, 2017. Colin Cameron Univ. of Calif.
More informationA Note on Data-Adaptive Bandwidth Selection for Sequential Kernel Smoothers
6th St.Petersburg Workshop on Simulation (2009) 1-3 A Note on Data-Adaptive Bandwidth Selection for Sequential Kernel Smoothers Ansgar Steland 1 Abstract Sequential kernel smoothers form a class of procedures
More informationMultivariate Regression
Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the
More informationPREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY
REVSTAT Statistical Journal Volume 7, Number 1, April 2009, 37 54 PREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY Authors: Germán Aneiros-Pérez Departamento de Matemáticas,
More informationSEMIPARAMETRIC ESTIMATION OF CONDITIONAL HETEROSCEDASTICITY VIA SINGLE-INDEX MODELING
Statistica Sinica 3 (013), 135-155 doi:http://dx.doi.org/10.5705/ss.01.075 SEMIPARAMERIC ESIMAION OF CONDIIONAL HEEROSCEDASICIY VIA SINGLE-INDEX MODELING Liping Zhu, Yuexiao Dong and Runze Li Shanghai
More informationNonparametric Identification of a Binary Random Factor in Cross Section Data - Supplemental Appendix
Nonparametric Identification of a Binary Random Factor in Cross Section Data - Supplemental Appendix Yingying Dong and Arthur Lewbel California State University Fullerton and Boston College July 2010 Abstract
More informationGoodness-of-fit tests for the cure rate in a mixture cure model
Biometrika (217), 13, 1, pp. 1 7 Printed in Great Britain Advance Access publication on 31 July 216 Goodness-of-fit tests for the cure rate in a mixture cure model BY U.U. MÜLLER Department of Statistics,
More informationNonparametric Inference via Bootstrapping the Debiased Estimator
Nonparametric Inference via Bootstrapping the Debiased Estimator Yen-Chi Chen Department of Statistics, University of Washington ICSA-Canada Chapter Symposium 2017 1 / 21 Problem Setup Let X 1,, X n be
More informationNADARAYA WATSON ESTIMATE JAN 10, 2006: version 2. Y ik ( x i
NADARAYA WATSON ESTIMATE JAN 0, 2006: version 2 DATA: (x i, Y i, i =,..., n. ESTIMATE E(Y x = m(x by n i= ˆm (x = Y ik ( x i x n i= K ( x i x EXAMPLES OF K: K(u = I{ u c} (uniform or box kernel K(u = u
More informationError distribution function for parametrically truncated and censored data
Error distribution function for parametrically truncated and censored data Géraldine LAURENT Jointly with Cédric HEUCHENNE QuantOM, HEC-ULg Management School - University of Liège Friday, 14 September
More informationRegression Discontinuity Design Econometric Issues
Regression Discontinuity Design Econometric Issues Brian P. McCall University of Michigan Texas Schools Project, University of Texas, Dallas November 20, 2009 1 Regression Discontinuity Design Introduction
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationNonparametric Density Estimation
Nonparametric Density Estimation Econ 690 Purdue University Justin L. Tobias (Purdue) Nonparametric Density Estimation 1 / 29 Density Estimation Suppose that you had some data, say on wages, and you wanted
More informationKernel Density Estimation
Kernel Density Estimation Univariate Density Estimation Suppose tat we ave a random sample of data X 1,..., X n from an unknown continuous distribution wit probability density function (pdf) f(x) and cumulative
More informationLocal regression I. Patrick Breheny. November 1. Kernel weighted averages Local linear regression
Local regression I Patrick Breheny November 1 Patrick Breheny STA 621: Nonparametric Statistics 1/27 Simple local models Kernel weighted averages The Nadaraya-Watson estimator Expected loss and prediction
More informationUNIT Define joint distribution and joint probability density function for the two random variables X and Y.
UNIT 4 1. Define joint distribution and joint probability density function for the two random variables X and Y. Let and represent the probability distribution functions of two random variables X and Y
More informationLecture 14 Simple Linear Regression
Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent
More informationA COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES
A COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES MICHAEL O HARA AND CHRISTOPHER F. PARMETER Abstract. This paper presents a Monte Carlo comparison of
More informationO Combining cross-validation and plug-in methods - for kernel density bandwidth selection O
O Combining cross-validation and plug-in methods - for kernel density selection O Carlos Tenreiro CMUC and DMUC, University of Coimbra PhD Program UC UP February 18, 2011 1 Overview The nonparametric problem
More informationGaussian Process Regression
Gaussian Process Regression 4F1 Pattern Recognition, 21 Carl Edward Rasmussen Department of Engineering, University of Cambridge November 11th - 16th, 21 Rasmussen (Engineering, Cambridge) Gaussian Process
More informationPreface. 1 Nonparametric Density Estimation and Testing. 1.1 Introduction. 1.2 Univariate Density Estimation
Preface Nonparametric econometrics has become one of the most important sub-fields in modern econometrics. The primary goal of this lecture note is to introduce various nonparametric and semiparametric
More informationOptimal Bandwidth Choice for the Regression Discontinuity Estimator
Optimal Bandwidth Choice for the Regression Discontinuity Estimator Guido Imbens and Karthik Kalyanaraman First Draft: June 8 This Draft: September Abstract We investigate the choice of the bandwidth for
More informationSemiparametric modeling and estimation of the dispersion function in regression
Semiparametric modeling and estimation of the dispersion function in regression Ingrid Van Keilegom Lan Wang September 4, 2008 Abstract Modeling heteroscedasticity in semiparametric regression can improve
More informationDirect Learning: Linear Classification. Donglin Zeng, Department of Biostatistics, University of North Carolina
Direct Learning: Linear Classification Logistic regression models for classification problem We consider two class problem: Y {0, 1}. The Bayes rule for the classification is I(P(Y = 1 X = x) > 1/2) so
More informationECONOMETRICS II (ECO 2401S) University of Toronto. Department of Economics. Spring 2013 Instructor: Victor Aguirregabiria
ECONOMETRICS II (ECO 2401S) University of Toronto. Department of Economics. Spring 2013 Instructor: Victor Aguirregabiria SOLUTION TO FINAL EXAM Friday, April 12, 2013. From 9:00-12:00 (3 hours) INSTRUCTIONS:
More informationIntroduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones
Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 Last week... supervised and unsupervised methods need adaptive
More informationAdaptive Kernel Estimation of The Hazard Rate Function
Adaptive Kernel Estimation of The Hazard Rate Function Raid Salha Department of Mathematics, Islamic University of Gaza, Palestine, e-mail: rbsalha@mail.iugaza.edu Abstract In this paper, we generalized
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More informationRegression Models - Introduction
Regression Models - Introduction In regression models, two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent variable,
More informationECON 4160, Autumn term Lecture 1
ECON 4160, Autumn term 2017. Lecture 1 a) Maximum Likelihood based inference. b) The bivariate normal model Ragnar Nymoen University of Oslo 24 August 2017 1 / 54 Principles of inference I Ordinary least
More informationNew Local Estimation Procedure for Nonparametric Regression Function of Longitudinal Data
ew Local Estimation Procedure for onparametric Regression Function of Longitudinal Data Weixin Yao and Runze Li The Pennsylvania State University Technical Report Series #0-03 College of Health and Human
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University
More informationOutline. Nature of the Problem. Nature of the Problem. Basic Econometrics in Transportation. Autocorrelation
1/30 Outline Basic Econometrics in Transportation Autocorrelation Amir Samimi What is the nature of autocorrelation? What are the theoretical and practical consequences of autocorrelation? Since the assumption
More informationTest for Discontinuities in Nonparametric Regression
Communications of the Korean Statistical Society Vol. 15, No. 5, 2008, pp. 709 717 Test for Discontinuities in Nonparametric Regression Dongryeon Park 1) Abstract The difference of two one-sided kernel
More informationEconometrics Honor s Exam Review Session. Spring 2012 Eunice Han
Econometrics Honor s Exam Review Session Spring 2012 Eunice Han Topics 1. OLS The Assumptions Omitted Variable Bias Conditional Mean Independence Hypothesis Testing and Confidence Intervals Homoskedasticity
More informationIntroduction An approximated EM algorithm Simulation studies Discussion
1 / 33 An Approximated Expectation-Maximization Algorithm for Analysis of Data with Missing Values Gong Tang Department of Biostatistics, GSPH University of Pittsburgh NISS Workshop on Nonignorable Nonresponse
More informationThe Priestley-Chao Estimator - Bias, Variance and Mean-Square Error
The Priestley-Chao Estimator - Bias, Variance and Mean-Square Error Bias, variance and mse properties In the previous section we saw that the eact mean and variance of the Pristley-Chao estimator ˆm()
More informationLecture 6: Discrete Choice: Qualitative Response
Lecture 6: Instructor: Department of Economics Stanford University 2011 Types of Discrete Choice Models Univariate Models Binary: Linear; Probit; Logit; Arctan, etc. Multinomial: Logit; Nested Logit; GEV;
More informationSemiparametric estimation of covariance matrices for longitudinal data
Semiparametric estimation of covariance matrices for longitudinal data Jianqing Fan and Yichao Wu Princeton University and North Carolina State University Abstract Estimation of longitudinal data covariance
More informationVariance Function Estimation in Multivariate Nonparametric Regression
Variance Function Estimation in Multivariate Nonparametric Regression T. Tony Cai 1, Michael Levine Lie Wang 1 Abstract Variance function estimation in multivariate nonparametric regression is considered
More informationSpatiotemporal Anatomical Atlas Building
Spatiotemporal Anatomical Atlas Building Population Shape Regression For Random Design Data Brad Davis 1, P. Thomas Fletcher 2, Elizabeth Bullitt 1, Sarang Joshi 2 1 The University of North Carolina at
More informationTitolo Smooth Backfitting with R
Rapporto n. 176 Titolo Smooth Backfitting with R Autori Alberto Arcagni, Luca Bagnato ottobre 2009 Dipartimento di Metodi Quantitativi per le Scienze Economiche ed Aziendali Università degli Studi di Milano
More informationCOMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017
COMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University FEATURE EXPANSIONS FEATURE EXPANSIONS
More informationDESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA
Statistica Sinica 18(2008), 515-534 DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA Kani Chen 1, Jianqing Fan 2 and Zhezhen Jin 3 1 Hong Kong University of Science and Technology,
More informationFinal Overview. Introduction to ML. Marek Petrik 4/25/2017
Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,
More informationThe Multivariate Normal Distribution 1
The Multivariate Normal Distribution 1 STA 302 Fall 2017 1 See last slide for copyright information. 1 / 40 Overview 1 Moment-generating Functions 2 Definition 3 Properties 4 χ 2 and t distributions 2
More informationLocal Polynomial Modelling and Its Applications
Local Polynomial Modelling and Its Applications J. Fan Department of Statistics University of North Carolina Chapel Hill, USA and I. Gijbels Institute of Statistics Catholic University oflouvain Louvain-la-Neuve,
More informationBootstrap of residual processes in regression: to smooth or not to smooth?
Bootstrap of residual processes in regression: to smooth or not to smooth? arxiv:1712.02685v1 [math.st] 7 Dec 2017 Natalie Neumeyer Ingrid Van Keilegom December 8, 2017 Abstract In this paper we consider
More informationChapter 2: Resampling Maarten Jansen
Chapter 2: Resampling Maarten Jansen Randomization tests Randomized experiment random assignment of sample subjects to groups Example: medical experiment with control group n 1 subjects for true medicine,
More informationSimple and Efficient Improvements of Multivariate Local Linear Regression
Journal of Multivariate Analysis Simple and Efficient Improvements of Multivariate Local Linear Regression Ming-Yen Cheng 1 and Liang Peng Abstract This paper studies improvements of multivariate local
More informationOptimal global rates of convergence for interpolation problems with random design
Optimal global rates of convergence for interpolation problems with random design Michael Kohler 1 and Adam Krzyżak 2, 1 Fachbereich Mathematik, Technische Universität Darmstadt, Schlossgartenstr. 7, 64289
More informationKernel density estimation
Kernel density estimation Patrick Breheny October 18 Patrick Breheny STA 621: Nonparametric Statistics 1/34 Introduction Kernel Density Estimation We ve looked at one method for estimating density: histograms
More informationClassification. Chapter Introduction. 6.2 The Bayes classifier
Chapter 6 Classification 6.1 Introduction Often encountered in applications is the situation where the response variable Y takes values in a finite set of labels. For example, the response Y could encode
More informationOptimal bandwidth selection for the fuzzy regression discontinuity estimator
Optimal bandwidth selection for the fuzzy regression discontinuity estimator Yoichi Arai Hidehiko Ichimura The Institute for Fiscal Studies Department of Economics, UCL cemmap working paper CWP49/5 Optimal
More informationUniversity, Tempe, Arizona, USA b Department of Mathematics and Statistics, University of New. Mexico, Albuquerque, New Mexico, USA
This article was downloaded by: [University of New Mexico] On: 27 September 2012, At: 22:13 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
More informationSimple Linear Regression
Simple Linear Regression September 24, 2008 Reading HH 8, GIll 4 Simple Linear Regression p.1/20 Problem Data: Observe pairs (Y i,x i ),i = 1,...n Response or dependent variable Y Predictor or independent
More information