Acceleration of some empirical means. Application to semiparametric regression
|
|
- Leo Casey
- 5 years ago
- Views:
Transcription
1 Acceleration of some empirical means. Application to semiparametric regression François Portier Université catholique de Louvain - ISBA November, In collaboration with Bernard Delyon
2 Regression model Y i = g(x i)+σ(x i)e i (X i) random i.i.d. with density f (X i) (e i) The functions g and σ are unknown Let Q R d bounded and L 2(Q) = {ψ : Purpose Q ψ(x)2 dx < + } Estimate c =< g, ψ >= g(x)ψ(x)dx Q (nonrandom design case treated by Donoho) 2 / 28
3 Plug-in estimates Plug-in of g is difficult Let ĝ such that a n(ĝ(x) g(x)) d Gaussian variable (e.g. NW, NN...) a n = o( n), but not tight, then n(< ĝ, ψ > < g, ψ >) = n < ĝ g, ψ > d Gaussian variable is diffucult to show. Plug-in of f may be better c =< g, ψ >= E [ ] Y ψ(x) f(x) ĉ = n 1 n i=1 Y iψ(x i) f(xi) 3 / 28
4 Fields of application Semiparametric estimation Dimension reduction g(x) = g 0(β T x) ADE: < g, ψ >= < g, ψ >= β (Vial, Härdle) Estimation of a location parameter in a regression (Vimond, Bercu) Curve estimation Orthogonal series or wavelet Kernel smoothing (Book of Härdle) K < g, ψ k=1 k > ψ k (y) K + g(y) approx.th. < g, K h ( y) > h 0 g(y) regular.th. 4 / 28
5 Integral approximation by kernel smoothing Asymptotic normality of ĉ Application to dimension reduction Proof, generalization, concluding remarks 5 / 28
6 Context Approximate the quantity ϕ(x)dx Q ϕ is known at the points X i (X i) is random (if X i is regular: quasi MC) Classical Montecarlo procedure n ( n 1 n i=1 ϕ(x i) f(x ϕ(x)dx i) Q ) d Gaussian variable Kernel smoothing n ( n 1 n i=1 ϕ(x i) f (i) (X i) Q ϕ(x)dx ) P 0 f (i) (x) = (nh d ) 1 n K(h 1 (X j x)) j i 6 / 28
7 Assumptions Nikol ski class H s, s = k + α, k N, 0 < α 1 (ϕ (l) (x + u) ϕ (l) (x)) 2 dx C u 2α l = (l 1,... l d ), l i k ( ψ is α-hölder inside Q s = min(1/2, α)) (A1) ϕ H s on R d and has compact support Q (A2) The r-th order derivatives of f are bounded (A3) For every x Q, f(x) b > 0 (A4) K symmetric with order r and K(x) C 1 exp( C 2 x ) 7 / 28
8 Theorem Assume (A1-A4), we have n ( n 1 n i=1 ϕ(x i) f (i) (X i) Q ϕ(x)dx ) = O P ( h s + n 1/2 h r + n 1/2 h d) (1) if the O P n + 0 Remarks Curse of dimensionality: r > d For r, s large, h opt n 1 r+d, f is undersmooth because h opt < n 1 2r+d Regularity of ϕ is not crucial Trimming method? the rate = n r d 2(r+d) (Stone) 8 / 28
9 Bandwidth choice Plug-in (Härdle, Hart, Marron and Tsybakov) Cross validation 9 / 28
10 Theorem v (i) (x) = ((n 1)(n 2)) 1 Assume (A1-A4) we have n (n 1 n i=1 if the O P n + 0 Remarks n (h d K(h 1 (x X j)) f (i) (x)) 2 j i ϕ(x i) ( v (i) (X i) ) 1 f (i) (X i) f (i) (X i) Q 2 Curse of dimensionality : r > 3d/4 ϕ(x)dx ( = O P h s + n 1/2 h r + n 1/2 h d/2 + n 1 h 3d/2) ( d) instead of O P h s + n 1/2 h r + n 1/2 h For r, s large, h opt n 1 r+d/2, the optimal rate = n r d/2 2(r+d/2) Leave-one out better than the classical ) 10 / 28
11 In practice Sample number = 20, h=n^1/3, Epanechnikov MC Boundary pb No boundary pb 11 / 28
12 In practice Sample number = 50, h=n^1/3, Epanechnikov MC Boundary pb No boundary pb 12 / 28
13 In practice Sample number = 100, h=n^1/3, Epanechnikov MC Boundary pb No boundary pb 13 / 28
14 In practice Sample number = 200, h=n^1/3, Epanechnikov MC Boundary pb No boundary pb 14 / 28
15 In practice Sample number = 500, h=n^1/3, Epanechnikov MC Boundary pb No boundary pb 15 / 28
16 Integral approximation by kernel smoothing Asymptotic normality of ĉ Application to dimension reduction Proof, generalization, concluding remarks 16 / 28
17 Assumptions (A2) The r-th order derivatives of f are bounded (A3) For every x Q, f(x) b > 0 (A4) K symmetric with order r and K(x) C 1 exp( C 2 x ) 17 / 28
18 Assumptions (A2) The r-th order derivatives of f are bounded (A3) For every x Q, f(x) b > 0 (A4) K symmetric with order r and K(x) C 1 exp( C 2 x ) (A5) ψ is Hölder on its support Q R d nonempty bounded and convex (A6) g is Hölder on Q and σ is bounded (A7) n 1/2 h r n + 0 and n 1/2 h d n / 28
19 Theorem Assume (A2-A7) we have n 1/2 (ĉ c) d N(0, v) where v is the variance of the random variable Y 1 g(x 1 ) f(x 1 ) ψ(x 1) Remarks Rates in root n The variance is smaller than when f is known Trimming method? (Härdle and Stoker (1989)) 18 / 28
20 Integral approximation by kernel smoothing Asymptotic normality of ĉ Application to dimension reduction Proof, generalization, concluding remarks 19 / 28
21 Single index model g(x) = g 0(β T x) vect(β) = E g(x) E x R d Estimation of g and then E[ g(x)] (Hristache, Juditsky et Spokoiny (2001), n consistent when dim(e) 4) Estimation of f E[ g(x)] = E[Y f(x) f(x) ] (Härdle et Stoker (1989), n consistent when dim(e) = 1) Idea < xψ(, t), g >= < ψ(, t), g >= β t E 20 / 28
22 Results β t = ψ(x, t) g(x)dx βt = n 1 n i=1 Y i xψ(x i, t) f(xi) Theorem Under (A3-A7) we have n( βt β t) d Gaussian for each t 21 / 28
23 corollaries Corollary Under (A3-A7), and some regularity conditions on xψ, we have n ( βt β t ) Gaussian process Corollary Under (A3-A7), and some regularity conditions on xψ, we have ( ) n β t βt t dt β tβ T d t dt Gaussian variable 22 / 28
24 Implémentation β t = ψ(x, t) g(x)dx βt = n 1 n i=1 Y i xψ(x i, t) f(xi) 1.Compute β t βt t dt 2. β eigenvectors associated to the d largest eigenvalues Radial kernel with order r Bandwidth h = 2n 1/(r+p) 23 / 28
25 ( ) π Y = cos 2 (X(1) µ) + 0.4e, X = d N(0, I) R 6, e = d N(0, 1), µ R. Model II Dist SIR SP SAVE µ Figure: Boxplot over 100 samples of the error of SIR, SAVE and our method, n = 150, µ varies 24 / 28
26 Adaptive method Estimation of f curse of dimensionality One has [ ] [ ] Y1( ψ)(ax) g(x)ψ(ax) AE = E E, f AX (AX) f AX (AX) for every vect(a) E in particular for A 0 = ββ T Procedure  ǫ = β βt + ǫi ǫ 0 25 / 28
27 Simulations Y = X (1) + 0.4e, X d = N(0, I) R 6, e d = N(0, 1) Model III Dist SPadap SIR final (23) Number of iterations Figure: Boxplots over 100 samples of the error for SIR and the adaptive method, n = / 28
28 Integral approximation by kernel smoothing Asymptotic normality of ĉ Application to dimension reduction Proof, generalization, concluding remarks 27 / 28
29 Proof and generalization Proof Taylor devlopment U-stat Kernel regularization Generalization Estimation of the functionals ϕ(x)η(f(x))dx 28 / 28
Integral approximation by kernel smoothing
Integral approximation by kernel smoothing François Portier Université catholique de Louvain - ISBA August, 29 2014 In collaboration with Bernard Delyon Topic of the talk: Given ϕ : R d R, estimation of
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationModelling Non-linear and Non-stationary Time Series
Modelling Non-linear and Non-stationary Time Series Chapter 2: Non-parametric methods Henrik Madsen Advanced Time Series Analysis September 206 Henrik Madsen (02427 Adv. TS Analysis) Lecture Notes September
More informationModel Specification Testing in Nonparametric and Semiparametric Time Series Econometrics. Jiti Gao
Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics Jiti Gao Department of Statistics School of Mathematics and Statistics The University of Western Australia Crawley
More informationDensity estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas
0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity
More informationMinimum Hellinger Distance Estimation in a. Semiparametric Mixture Model
Minimum Hellinger Distance Estimation in a Semiparametric Mixture Model Sijia Xiang 1, Weixin Yao 1, and Jingjing Wu 2 1 Department of Statistics, Kansas State University, Manhattan, Kansas, USA 66506-0802.
More informationO Combining cross-validation and plug-in methods - for kernel density bandwidth selection O
O Combining cross-validation and plug-in methods - for kernel density selection O Carlos Tenreiro CMUC and DMUC, University of Coimbra PhD Program UC UP February 18, 2011 1 Overview The nonparametric problem
More informationPhysics 443, Solutions to PS 1 1
Physics 443, Solutions to PS. Griffiths.9 For Φ(x, t A exp[ a( mx + it], we need that + h Φ(x, t dx. Using the known result of a Gaussian intergral + exp[ ax ]dx /a, we find that: am A h. ( The Schrödinger
More informationDEPARTMENT MATHEMATIK ARBEITSBEREICH MATHEMATISCHE STATISTIK UND STOCHASTISCHE PROZESSE
Estimating the error distribution in nonparametric multiple regression with applications to model testing Natalie Neumeyer & Ingrid Van Keilegom Preprint No. 2008-01 July 2008 DEPARTMENT MATHEMATIK ARBEITSBEREICH
More informationOPTIMAL POINTWISE ADAPTIVE METHODS IN NONPARAMETRIC ESTIMATION 1
The Annals of Statistics 1997, Vol. 25, No. 6, 2512 2546 OPTIMAL POINTWISE ADAPTIVE METHODS IN NONPARAMETRIC ESTIMATION 1 By O. V. Lepski and V. G. Spokoiny Humboldt University and Weierstrass Institute
More informationVariance Function Estimation in Multivariate Nonparametric Regression
Variance Function Estimation in Multivariate Nonparametric Regression T. Tony Cai 1, Michael Levine Lie Wang 1 Abstract Variance function estimation in multivariate nonparametric regression is considered
More informationBickel Rosenblatt test
University of Latvia 28.05.2011. A classical Let X 1,..., X n be i.i.d. random variables with a continuous probability density function f. Consider a simple hypothesis H 0 : f = f 0 with a significance
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationTime Series and Forecasting Lecture 4 NonLinear Time Series
Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/100 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression
More informationOn variable bandwidth kernel density estimation
JSM 04 - Section on Nonparametric Statistics On variable bandwidth kernel density estimation Janet Nakarmi Hailin Sang Abstract In this paper we study the ideal variable bandwidth kernel estimator introduced
More informationKernel Density Estimation
EECS 598: Statistical Learning Theory, Winter 2014 Topic 19 Kernel Density Estimation Lecturer: Clayton Scott Scribe: Yun Wei, Yanzhen Deng Disclaimer: These notes have not been subjected to the usual
More informationIntroduction to Regression
Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric
More informationEstimating a frontier function using a high-order moments method
1/ 16 Estimating a frontier function using a high-order moments method Gilles STUPFLER (University of Nottingham) Joint work with Stéphane GIRARD (INRIA Rhône-Alpes) and Armelle GUILLOU (Université de
More informationNonparametric Inference via Bootstrapping the Debiased Estimator
Nonparametric Inference via Bootstrapping the Debiased Estimator Yen-Chi Chen Department of Statistics, University of Washington ICSA-Canada Chapter Symposium 2017 1 / 21 Problem Setup Let X 1,, X n be
More informationNonparametric estimation using wavelet methods. Dominique Picard. Laboratoire Probabilités et Modèles Aléatoires Université Paris VII
Nonparametric estimation using wavelet methods Dominique Picard Laboratoire Probabilités et Modèles Aléatoires Université Paris VII http ://www.proba.jussieu.fr/mathdoc/preprints/index.html 1 Nonparametric
More informationOrdinary Differential Equations II
Ordinary Differential Equations II February 9 217 Linearization of an autonomous system We consider the system (1) x = f(x) near a fixed point x. As usual f C 1. Without loss of generality we assume x
More informationThe memory centre IMUJ PREPRINT 2012/03. P. Spurek
The memory centre IMUJ PREPRINT 202/03 P. Spurek Faculty of Mathematics and Computer Science, Jagiellonian University, Łojasiewicza 6, 30-348 Kraków, Poland J. Tabor Faculty of Mathematics and Computer
More informationFunction Spaces. 1 Hilbert Spaces
Function Spaces A function space is a set of functions F that has some structure. Often a nonparametric regression function or classifier is chosen to lie in some function space, where the assume structure
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo Han Liu John Lafferty Larry Wasserman Statistics Department Computer Science Department Machine Learning Department Carnegie Mellon
More informationThe high order moments method in endpoint estimation: an overview
1/ 33 The high order moments method in endpoint estimation: an overview Gilles STUPFLER (Aix Marseille Université) Joint work with Stéphane GIRARD (INRIA Rhône-Alpes) and Armelle GUILLOU (Université de
More informationSufficient Dimension Reduction via Direct Estimation of the Gradients of Logarithmic Conditional Densities
JMLR: Workshop and Conference Proceedings 30:1 16, 2015 ACML 2015 Sufficient Dimension Reduction via Direct Estimation of the Gradients of Logarithmic Conditional Densities Hiroaki Sasaki Graduate School
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression
More informationWavelets in Scattering Calculations
Wavelets in Scattering Calculations W. P., Brian M. Kessler, Gerald L. Payne polyzou@uiowa.edu The University of Iowa Wavelets in Scattering Calculations p.1/43 What are Wavelets? Orthonormal basis functions.
More informationNonparametric confidence intervals. for receiver operating characteristic curves
Nonparametric confidence intervals for receiver operating characteristic curves Peter G. Hall 1, Rob J. Hyndman 2, and Yanan Fan 3 5 December 2003 Abstract: We study methods for constructing confidence
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University
More informationMinimax Estimation of a nonlinear functional on a structured high-dimensional model
Minimax Estimation of a nonlinear functional on a structured high-dimensional model Eric Tchetgen Tchetgen Professor of Biostatistics and Epidemiologic Methods, Harvard U. (Minimax ) 1 / 38 Outline Heuristics
More informationBoundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta
Boundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta November 29, 2007 Outline Overview of Kernel Density
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More informationNonparametric Econometrics
Applied Microeconometrics with Stata Nonparametric Econometrics Spring Term 2011 1 / 37 Contents Introduction The histogram estimator The kernel density estimator Nonparametric regression estimators Semi-
More informationStatistica Sinica Preprint No: SS
Statistica Sinica Preprint No: SS-017-0013 Title A Bootstrap Method for Constructing Pointwise and Uniform Confidence Bands for Conditional Quantile Functions Manuscript ID SS-017-0013 URL http://wwwstatsinicaedutw/statistica/
More informationAnalysis methods of heavy-tailed data
Institute of Control Sciences Russian Academy of Sciences, Moscow, Russia February, 13-18, 2006, Bamberg, Germany June, 19-23, 2006, Brest, France May, 14-19, 2007, Trondheim, Norway PhD course Chapter
More informationMonte Carlo Solution of Integral Equations
1 Monte Carlo Solution of Integral Equations of the second kind Adam M. Johansen a.m.johansen@warwick.ac.uk www2.warwick.ac.uk/fac/sci/statistics/staff/academic/johansen/talks/ March 7th, 2011 MIRaW Monte
More informationThe Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space
The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space Robert Jenssen, Deniz Erdogmus 2, Jose Principe 2, Torbjørn Eltoft Department of Physics, University of Tromsø, Norway
More informationSemiparametric modeling and estimation of the dispersion function in regression
Semiparametric modeling and estimation of the dispersion function in regression Ingrid Van Keilegom Lan Wang September 4, 2008 Abstract Modeling heteroscedasticity in semiparametric regression can improve
More informationA review of some semiparametric regression models with application to scoring
A review of some semiparametric regression models with application to scoring Jean-Loïc Berthet 1 and Valentin Patilea 2 1 ENSAI Campus de Ker-Lann Rue Blaise Pascal - BP 37203 35172 Bruz cedex, France
More informationMath Camp Notes: Everything Else
Math Camp Notes: Everything Else Systems of Dierential Equations Consider the general two-equation system of dierential equations: Steady States ẋ = f(x, y ẏ = g(x, y Just as before, we can nd the steady
More informationA Note on Data-Adaptive Bandwidth Selection for Sequential Kernel Smoothers
6th St.Petersburg Workshop on Simulation (2009) 1-3 A Note on Data-Adaptive Bandwidth Selection for Sequential Kernel Smoothers Ansgar Steland 1 Abstract Sequential kernel smoothers form a class of procedures
More informationFractal functional regression for classification of gene expression data by wavelets
Fractal functional regression for classification of gene expression data by wavelets Margarita María Rincón 1 and María Dolores Ruiz-Medina 2 1 University of Granada Campus Fuente Nueva 18071 Granada,
More informationLocal linear multiple regression with variable. bandwidth in the presence of heteroscedasticity
Local linear multiple regression with variable bandwidth in the presence of heteroscedasticity Azhong Ye 1 Rob J Hyndman 2 Zinai Li 3 23 January 2006 Abstract: We present local linear estimator with variable
More informationNonparametric Density Estimation (Multidimension)
Nonparametric Density Estimation (Multidimension) Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann February 19, 2007 Setup One-dimensional
More informationLocal Polynomial Regression
VI Local Polynomial Regression (1) Global polynomial regression We observe random pairs (X 1, Y 1 ),, (X n, Y n ) where (X 1, Y 1 ),, (X n, Y n ) iid (X, Y ). We want to estimate m(x) = E(Y X = x) based
More informationOptimal Estimation of a Nonsmooth Functional
Optimal Estimation of a Nonsmooth Functional T. Tony Cai Department of Statistics The Wharton School University of Pennsylvania http://stat.wharton.upenn.edu/ tcai Joint work with Mark Low 1 Question Suppose
More informationInverse problems in statistics
Inverse problems in statistics Laurent Cavalier (Université Aix-Marseille 1, France) Yale, May 2 2011 p. 1/35 Introduction There exist many fields where inverse problems appear Astronomy (Hubble satellite).
More informationWavelet Analysis. Willy Hereman. Department of Mathematical and Computer Sciences Colorado School of Mines Golden, CO Sandia Laboratories
Wavelet Analysis Willy Hereman Department of Mathematical and Computer Sciences Colorado School of Mines Golden, CO 8040-887 Sandia Laboratories December 0, 998 Coordinate-Coordinate Formulations CC and
More informationBy 3DYHOýtåHN, Wolfgang Härdle
No. 2005 31 ROBUST ESTIMATION OF DIMENSION REDUCTION SPACE By 3DYHOýtåHN, Wolfgang Härdle February 2005 ISSN 0924-7815 Robust estimation of dimension reduction space P. Čížek a and W. Härdle b a Department
More informationQuantitative Economics for the Evaluation of the European Policy. Dipartimento di Economia e Management
Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti 1 Davide Fiaschi 2 Angela Parenti 3 9 ottobre 2015 1 ireneb@ec.unipi.it. 2 davide.fiaschi@unipi.it.
More informationKernel Density Estimation
Kernel Density Estimation and Application in Discriminant Analysis Thomas Ledl Universität Wien Contents: Aspects of Application observations: 0 Which distribution? 0?? 0.0 0. 0. 0. 0.0 0. 0. 0 0 0.0
More informationThermodynamic limit for a system of interacting fermions in a random medium. Pieces one-dimensional model
Thermodynamic limit for a system of interacting fermions in a random medium. Pieces one-dimensional model Nikolaj Veniaminov (in collaboration with Frédéric Klopp) CEREMADE, University of Paris IX Dauphine
More informationS6880 #13. Variance Reduction Methods
S6880 #13 Variance Reduction Methods 1 Variance Reduction Methods Variance Reduction Methods 2 Importance Sampling Importance Sampling 3 Control Variates Control Variates Cauchy Example Revisited 4 Antithetic
More informationAdaptive Kernel Estimation of The Hazard Rate Function
Adaptive Kernel Estimation of The Hazard Rate Function Raid Salha Department of Mathematics, Islamic University of Gaza, Palestine, e-mail: rbsalha@mail.iugaza.edu Abstract In this paper, we generalized
More informationCS 556: Computer Vision. Lecture 21
CS 556: Computer Vision Lecture 21 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu 1 Meanshift 2 Meanshift Clustering Assumption: There is an underlying pdf governing data properties in R Clustering
More informationNonparametric Estimation of Functional-Coefficient Autoregressive Models
Nonparametric Estimation of Functional-Coefficient Autoregressive Models PEDRO A. MORETTIN and CHANG CHIANN Department of Statistics, University of São Paulo Introduction Nonlinear Models: - Exponential
More informationECO Class 6 Nonparametric Econometrics
ECO 523 - Class 6 Nonparametric Econometrics Carolina Caetano Contents 1 Nonparametric instrumental variable regression 1 2 Nonparametric Estimation of Average Treatment Effects 3 2.1 Asymptotic results................................
More informationTD 1: Hilbert Spaces and Applications
Université Paris-Dauphine Functional Analysis and PDEs Master MMD-MA 2017/2018 Generalities TD 1: Hilbert Spaces and Applications Exercise 1 (Generalized Parallelogram law). Let (H,, ) be a Hilbert space.
More information4 Invariant Statistical Decision Problems
4 Invariant Statistical Decision Problems 4.1 Invariant decision problems Let G be a group of measurable transformations from the sample space X into itself. The group operation is composition. Note that
More informationTESTING REGRESSION MONOTONICITY IN ECONOMETRIC MODELS
TESTING REGRESSION MONOTONICITY IN ECONOMETRIC MODELS DENIS CHETVERIKOV Abstract. Monotonicity is a key qualitative prediction of a wide array of economic models derived via robust comparative statics.
More informationInstance-based Learning CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016
Instance-based Learning CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Outline Non-parametric approach Unsupervised: Non-parametric density estimation Parzen Windows Kn-Nearest
More informationIntroduction to Curve Estimation
Introduction to Curve Estimation Density 0.000 0.002 0.004 0.006 700 800 900 1000 1100 1200 1300 Wilcoxon score Michael E. Tarter & Micheal D. Lock Model-Free Curve Estimation Monographs on Statistics
More informationTheory of Positive Definite Kernel and Reproducing Kernel Hilbert Space
Theory of Positive Definite Kernel and Reproducing Kernel Hilbert Space Statistical Inference with Reproducing Kernel Hilbert Space Kenji Fukumizu Institute of Statistical Mathematics, ROIS Department
More informationSmoothness Adaptive Average Derivative Estimation
Econometrics Journal 009, volume 10, pp. 1 3. Article No. ectj?????? Smoothness Adaptive Average Derivative Estimation Marcia M.A. Schafgans and Victoria Zinde-Walsh Department of Economics, London School
More informationStatistical inference on Lévy processes
Alberto Coca Cabrero University of Cambridge - CCA Supervisors: Dr. Richard Nickl and Professor L.C.G.Rogers Funded by Fundación Mutua Madrileña and EPSRC MASDOC/CCA student workshop 2013 26th March Outline
More informationIntroduction. Linear Regression. coefficient estimates for the wage equation: E(Y X) = X 1 β X d β d = X β
Introduction - Introduction -2 Introduction Linear Regression E(Y X) = X β +...+X d β d = X β Example: Wage equation Y = log wages, X = schooling (measured in years), labor market experience (measured
More information7 Semiparametric Estimation of Additive Models
7 Semiparametric Estimation of Additive Models Additive models are very useful for approximating the high-dimensional regression mean functions. They and their extensions have become one of the most widely
More informationg(x) = P (y) Proof. This is true for n = 0. Assume by the inductive hypothesis that g (n) (0) = 0 for some n. Compute g (n) (h) g (n) (0)
Mollifiers and Smooth Functions We say a function f from C is C (or simply smooth) if all its derivatives to every order exist at every point of. For f : C, we say f is C if all partial derivatives to
More informationLog Concavity and Density Estimation
Log Concavity and Density Estimation Lutz Dümbgen, Kaspar Rufibach, André Hüsler (Universität Bern) Geurt Jongbloed (Amsterdam) Nicolai Bissantz (Göttingen) I. A Consulting Case II. III. IV. Log Concave
More informationUnsupervised Learning Techniques Class 07, 1 March 2006 Andrea Caponnetto
Unsupervised Learning Techniques 9.520 Class 07, 1 March 2006 Andrea Caponnetto About this class Goal To introduce some methods for unsupervised learning: Gaussian Mixtures, K-Means, ISOMAP, HLLE, Laplacian
More informationQuantum Mechanics Exercises and solutions
Quantum Mechanics Exercises and solutions P.J. Mulders Department of Physics and Astronomy, Faculty of Sciences, Vrije Universiteit Amsterdam De Boelelaan 181, 181 HV Amsterdam, the Netherlands email:
More informationSimple and Honest Confidence Intervals in Nonparametric Regression
Simple and Honest Confidence Intervals in Nonparametric Regression Timothy B. Armstrong Yale University Michal Kolesár Princeton University June, 206 Abstract We consider the problem of constructing honest
More informationFast learning rates for plug-in classifiers under the margin condition
Fast learning rates for plug-in classifiers under the margin condition Jean-Yves Audibert 1 Alexandre B. Tsybakov 2 1 Certis ParisTech - Ecole des Ponts, France 2 LPMA Université Pierre et Marie Curie,
More informationAsymptotics of minimax stochastic programs
Asymptotics of minimax stochastic programs Alexander Shapiro Abstract. We discuss in this paper asymptotics of the sample average approximation (SAA) of the optimal value of a minimax stochastic programming
More informationApproximating the Best Linear Unbiased Estimator of Non-Gaussian Signals with Gaussian Noise
IEICE Transactions on Information and Systems, vol.e91-d, no.5, pp.1577-1580, 2008. 1 Approximating the Best Linear Unbiased Estimator of Non-Gaussian Signals with Gaussian Noise Masashi Sugiyama (sugi@cs.titech.ac.jp)
More informationNonparametric Regression Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction
Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Univariate Kernel Regression The relationship between two variables, X and Y where m(
More informationLocal Modelling of Nonlinear Dynamic Systems Using Direct Weight Optimization
Local Modelling of Nonlinear Dynamic Systems Using Direct Weight Optimization Jacob Roll, Alexander Nazin, Lennart Ljung Division of Automatic Control Department of Electrical Engineering Linköpings universitet,
More informationGaussian processes for inference in stochastic differential equations
Gaussian processes for inference in stochastic differential equations Manfred Opper, AI group, TU Berlin November 6, 2017 Manfred Opper, AI group, TU Berlin (TU Berlin) inference in SDE November 6, 2017
More informationNonparametric Density Estimation
Nonparametric Density Estimation Econ 690 Purdue University Justin L. Tobias (Purdue) Nonparametric Density Estimation 1 / 29 Density Estimation Suppose that you had some data, say on wages, and you wanted
More informationJEREMY TAYLOR S CONTRIBUTIONS TO TRANSFORMATION MODEL
1 / 25 JEREMY TAYLOR S CONTRIBUTIONS TO TRANSFORMATION MODELS DEPT. OF STATISTICS, UNIV. WISCONSIN, MADISON BIOMEDICAL STATISTICAL MODELING. CELEBRATION OF JEREMY TAYLOR S OF 60TH BIRTHDAY. UNIVERSITY
More informationFinite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product
Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )
More informationIntroduction to Nonparametric and Semiparametric Estimation. Good when there are lots of data and very little prior information on functional form.
1 Introduction to Nonparametric and Semiparametric Estimation Good when there are lots of data and very little prior information on functional form. Examples: y = f(x) + " (nonparametric) y = z 0 + f(x)
More informationSobolev Spaces. Chapter 10
Chapter 1 Sobolev Spaces We now define spaces H 1,p (R n ), known as Sobolev spaces. For u to belong to H 1,p (R n ), we require that u L p (R n ) and that u have weak derivatives of first order in L p
More informationThree Papers by Peter Bickel on Nonparametric Curve Estimation
Three Papers by Peter Bickel on Nonparametric Curve Estimation Hans-Georg Müller 1 ABSTRACT The following is a brief review of three landmark papers of Peter Bickel on theoretical and methodological aspects
More informationDiscussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon
Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics
More informationEfficient Estimation for the Partially Linear Models with Random Effects
A^VÇÚO 1 33 ò 1 5 Ï 2017 c 10 Chinese Journal of Applied Probability and Statistics Oct., 2017, Vol. 33, No. 5, pp. 529-537 doi: 10.3969/j.issn.1001-4268.2017.05.009 Efficient Estimation for the Partially
More informationDensity estimators for the convolution of discrete and continuous random variables
Density estimators for the convolution of discrete and continuous random variables Ursula U Müller Texas A&M University Anton Schick Binghamton University Wolfgang Wefelmeyer Universität zu Köln Abstract
More informationNonparametric Estimation of Regression Functions In the Presence of Irrelevant Regressors
Nonparametric Estimation of Regression Functions In the Presence of Irrelevant Regressors Peter Hall, Qi Li, Jeff Racine 1 Introduction Nonparametric techniques robust to functional form specification.
More informationOn Efficiency of the Plug-in Principle for Estimating Smooth Integrated Functionals of a Nonincreasing Density
On Efficiency of the Plug-in Principle for Estimating Smooth Integrated Functionals of a Nonincreasing Density arxiv:188.7915v2 [math.st] 12 Feb 219 Rajarshi Mukherjee and Bodhisattva Sen Department of
More informationKullback-Leibler Designs
Kullback-Leibler Designs Astrid JOURDAN Jessica FRANCO Contents Contents Introduction Kullback-Leibler divergence Estimation by a Monte-Carlo method Design comparison Conclusion 2 Introduction Computer
More informationDIFFERENT KINDS OF ESTIMATORS OF THE MEAN DENSITY OF RANDOM CLOSED SETS: THEORETICAL RESULTS AND NUMERICAL EXPERIMENTS.
DIFFERENT KINDS OF ESTIMATORS OF THE MEAN DENSITY OF RANDOM CLOSED SETS: THEORETICAL RESULTS AND NUMERICAL EXPERIMENTS Elena Villa Dept. of Mathematics Università degli Studi di Milano Toronto May 22,
More informationMinimax lower bounds I
Minimax lower bounds I Kyoung Hee Kim Sungshin University 1 Preliminaries 2 General strategy 3 Le Cam, 1973 4 Assouad, 1983 5 Appendix Setting Family of probability measures {P θ : θ Θ} on a sigma field
More informationAdditive Models: Extensions and Related Models.
Additive Models: Extensions and Related Models. Enno Mammen Byeong U. Park Melanie Schienle August 8, 202 Abstract We give an overview over smooth backfitting type estimators in additive models. Moreover
More informationECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd
ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation Petra E. Todd Fall, 2014 2 Contents 1 Review of Stochastic Order Symbols 1 2 Nonparametric Density Estimation 3 2.1 Histogram
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationSmooth simultaneous confidence bands for cumulative distribution functions
Journal of Nonparametric Statistics, 2013 Vol. 25, No. 2, 395 407, http://dx.doi.org/10.1080/10485252.2012.759219 Smooth simultaneous confidence bands for cumulative distribution functions Jiangyan Wang
More informationON THE CHOICE OF TEST STATISTIC FOR CONDITIONAL MOMENT INEQUALITES. Timothy B. Armstrong. October 2014 Revised July 2017
ON THE CHOICE OF TEST STATISTIC FOR CONDITIONAL MOMENT INEQUALITES By Timothy B. Armstrong October 2014 Revised July 2017 COWLES FOUNDATION DISCUSSION PAPER NO. 1960R2 COWLES FOUNDATION FOR RESEARCH IN
More informationThe Kernel Trick, Gram Matrices, and Feature Extraction. CS6787 Lecture 4 Fall 2017
The Kernel Trick, Gram Matrices, and Feature Extraction CS6787 Lecture 4 Fall 2017 Momentum for Principle Component Analysis CS6787 Lecture 3.1 Fall 2017 Principle Component Analysis Setting: find the
More information