Nonparametric Density Estimation (Multidimension)
|
|
- Daniel Moses Day
- 5 years ago
- Views:
Transcription
1 Nonparametric Density Estimation (Multidimension) Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann February 19, 2007
2 Setup One-dimensional estimation Multivariate estimation Consider a d-dimensional data set with sample size n X i = X i1. X id, i = 1,..., n. Goal: Estimate the density f of X = (X 1,..., X d ) T f (x) = f (x 1,..., x d )
3 Multivariate kernel density estimator Kernel density estimator in d-dimensions ˆf h (x) = 1 n = 1 n n i=1 n i=1 ( 1 x h d K Xi h 1 h d K ) ( x1 X i1 h,..., x ) d X id h where K is a multivariate kernel function with d arguments. Note: h is the same for each components.
4 Multivariate kernel density estimator Extension: Bandwidths: h = (h 1,..., h d ) T ˆf h (x) = 1 n n i=1 ( 1 x1 X i1 K,..., x ) d X id h 1...h d h 1 h d
5 Kernel function What form should the multidim. kernel K(u) = K(u 1,..., u d ) take? Multiplicative kernel: K(u) = K(u 1 )... K(u d ) where K is a univariate kernel function. ˆf h (x) = 1 n ( 1 x1 X i1 K,..., x ) d X id n h 1...h d h 1 h d i=1 = 1 n d ( ) 1 xj X ij K n h j i=1 j=1 h j Note: Contributions to the sum only in the cube: X i1 [x 1 h 1, x 1 + h 1 ),..., X id [x d h d, x d + h d )
6 Kernel function Spherical/radial-symmetric kernel: or K(u) K( u ) K(u) = K( u ) R K( u ) d where u = u T u. (Exercise 3.13) The multivariate Epanechnikov (spherical): K(u) (1 u T u)1 (u T u 1) The multivariate Epanechnikov (multiplicative): K(u) = ( ) 3 d (1 u 2 4 1)1 ( u1 1)...(1 ud 2 )1 ( u d 1)
7 Kernel function Epanechnikov kernel function Equal bandwidth in each direction: h = (h 1, h 2 ) T = (1, 1) T
8 Kernel function Epanechnikov kernel function Different bandwidth in each direction: h = (h 1, h 2 ) T = (1, 0.5) T
9 Multivariate kernel density estimator The general form for the multivariate density estimator with bandwidth matrix H (nonsingular) ˆf H (x) = 1 n = 1 n n i=1 1 det(h) K ( H 1 (x X i ) ) n K H (x X i ) i=1 where K H ( ) = 1 det(h) K(H 1 )
10 Multivariate kernel density estimator The bandwidth matrix includes all simpler cases. Equal bandwidth h: H = hi d where I d is the d d identity matrix. Different bandwidths h 1,..., h d : H = diag(h 1,..., h d )
11 Multivariate kernel density estimator What effect has the off-diagonal elements? Rule-of-Thumb: Use a bandwidth matrix proportional to ˆΣ 1 2, where ˆΣ is the covariance matrix of the data. Such a bandwidth corresponds to a transformation of the data, so that they have an identity covariance matrix, ie. we can use bandwidths matrics to adjust for correlation between the components.
12 Kernel function Epanechnikov kernel function Bandwidth matrix: ( H = )
13 Properties of the kernel function K is a density function R d K(u) du = 1 and K(u) 0 K is symmetric R d uk(u) du = 0 d K has a second moment (matrix) uu T K(u) du = µ 2 (K)I d R d where I d denotes the d d identity matrix K has a kernel norm K 2 2 = K 2 (u) du
14 Properties of the kernel function K is a density function. Therefore is also ˆf H a density function ˆf H (x) dx = 1 The estimate is consistent in any point x ˆf H (x) = 1 n n K H (X i x) P f (x) i=1
15 Statistical Properties Bias: ) E (ˆfH (x) f (x) 1 2 µ 2(K)tr{H T H f (x)h} Variance: ) V (ˆf H (x) AMISE: AMISE(H) = 1 4 µ2 2(K) 1 n det(h) K 2 2f (x) tr{h T H f (x)h} 2 dx + 1 n det(h) K 2 2 where H f is the Hessian matrix and K 2 2 squared L 2 -norm af K. is the d dimensional
16 Special case Univariate case: For d = 1 we obtain H = h, K = K, H f (x) = f (x) Bias: ) E (ˆf H (x) f (x) 1 2 µ 2(K)tr{H T H f (x)h} 1 2 µ 2(K)h 2 f (x) Variance: ) V (ˆfH (x) 1 n det(h) K 2 2f (x) 1 nh K 2 2f (x)
17 Bandwidth selection AMISE optimal bandwidth: We have a bias-variance trade-off which is solved in the AMISE optimal bandwidth. h is a scalar, H = hh 0 and det(h 0 ) = 1, then AMISE(H) = 1 [ ] 2 4 h4 µ 2 2(K) tr{h T 1 0 H f (x)h 0 } dx + nh d K 2 2 Then the optimal bandwidth and the optimal AMISE are h opt n 1/(4+d), AMISE(h opt H 0 ) n 4/(4+d) Note: The multivariate density estimator has a slower rate of convergens compared to the univariate one. H = hi d and fix sample size n: The AMISE optimal bandwidth larger in higher dimensions.
18 Bandwidth selection Bandwidth selection: Plug-in method (rule-of-thumb, generalized Silvermann rule-of-thumb) Cross-validation method
19 Bandwidth selection Plug-in method Idea: Optimize AMISE under the assumption that f is multivariate normal distribution N d (µ, Σ) and K is a multivariate Gaussian, ie. N d (0, I), then µ 2 (K) = 1 K 2 2 = 2 d π d/2 Then = tr{h T H f (x)h} 2 dx 1 2 d+2 π d/2 det(σ) 1/2 [2tr(HT Σ 1 H) 2 + {tr(h T Σ 1 H)} 2 ]
20 Bandwidth selection Simple case: H = diag(h 1,..., h d ) and Σ = diag(σ 1,..., σ d ), then ( ) 4 1/(d+4) h j = n 1/(d+4) σ j d + 2 }{{} C Silverman s rule-of-thumb (d = 1): ( 4ˆσ 5 ĥ rot = 3n ) 1/5
21 Bandwidth selection Replace σ j with ˆσ j and notice that C always is between (d = 11) and (d = 1): Scott s rule ĥ j = n 1/(d+4)ˆσ j It is not possibel to derive the rule-of-thumb in the general case, but it might be a good idea to choose the bandwidth matrix proportional to the covariance matrix. Generalization of Scott s rule: 1/(d+4) Ĥ = n ˆΣ1/2
22 Bandwidth selection Cross-validation: ISE(H) = = ) 2 (ˆfH (x) f (x) dx ˆf 2 H (x) dx }{{} Cal. from data Estimate of the expectation + Eˆf H (X) = 1 n f 2 (x) dx } {{ } Ignore n ˆf H, i (X i ) i=1 ) 2 (ˆfH f (x) dx }{{} =Eˆf H (X) where the multivariate version of the leave-one-out estimator is ˆf H, i (x) = 1 n K H (X j x) n 1 j=1,j i
23 Bandwidth selection Multivariate cross-validation criterion: CV(H) = 1 n 2 det(h) 2 n(n 1) n i=1 j=1 n n K K { H 1 (X j X i ) } n i=1 j=1,j i K H (X j X i ) Note: The bandwidths is a d d matrix H which means we have to minimize over d(d+1) 2 parameters. Even if H is diagonal matrix, we have a d-dimensional optimization problem.
24 Canonical bandwidths The canonical bandwidth of kernel j { } K δ j 2 1/(d+4) = 2 µ 2 (K ) 2 Therefore where AMISE(H j, K j ) = AMISE(H i, K i ) H i = δi δ j Hj
25 Canonical bandwidths Example: Adjust from Gaussian to Quartic product kernel d δ G δ Q δ Q /δ G
26 Graphical representation Example: Two-dimensions Est-West German migration intention in Spring Explanatory variables: Age and household income Two-dimensional nonparametric density estimate ˆf h (x) = ˆf h (x 1, x 2 ) where the bandwidth matrix H = diag(h)
27 Graphical representation Contour plot
28 Graphical representation Example: Three-dimensions How can we display three- or even higher diemsional density estimates? Hold one variable fix and plot the density function depending on the other variables. For three-dimensions we have x 1, x 2 vs. ˆf h (x 1, x 2, x 3 ) x 1, x 3 vs. ˆf h (x 1, x 2, x 3 ) x 2, x 3 vs. ˆf h (x 1, x 2, x 3 )
29 Graphical representation Example: Three-dimensions Credit scoring sample. Explanatory variables: Duration of the credit, household income and age.
30 Graphical representation Contour plot
Histogram Härdle, Müller, Sperlich, Werwatz, 1995, Nonparametric and Semiparametric Models, An Introduction
Härdle, Müller, Sperlich, Werwatz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Construction X 1,..., X n iid r.v. with (unknown) density, f. Aim: Estimate the density
More informationNonparametric Regression Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction
Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Univariate Kernel Regression The relationship between two variables, X and Y where m(
More informationIntroduction. Linear Regression. coefficient estimates for the wage equation: E(Y X) = X 1 β X d β d = X β
Introduction - Introduction -2 Introduction Linear Regression E(Y X) = X β +...+X d β d = X β Example: Wage equation Y = log wages, X = schooling (measured in years), labor market experience (measured
More informationQuantitative Economics for the Evaluation of the European Policy. Dipartimento di Economia e Management
Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti 1 Davide Fiaschi 2 Angela Parenti 3 9 ottobre 2015 1 ireneb@ec.unipi.it. 2 davide.fiaschi@unipi.it.
More informationModelling Non-linear and Non-stationary Time Series
Modelling Non-linear and Non-stationary Time Series Chapter 2: Non-parametric methods Henrik Madsen Advanced Time Series Analysis September 206 Henrik Madsen (02427 Adv. TS Analysis) Lecture Notes September
More informationKernel density estimation
Kernel density estimation Patrick Breheny October 18 Patrick Breheny STA 621: Nonparametric Statistics 1/34 Introduction Kernel Density Estimation We ve looked at one method for estimating density: histograms
More informationNonparametric Density Estimation
Nonparametric Density Estimation Econ 690 Purdue University Justin L. Tobias (Purdue) Nonparametric Density Estimation 1 / 29 Density Estimation Suppose that you had some data, say on wages, and you wanted
More informationKernel density estimation for heavy-tailed distributions...
Kernel density estimation for heavy-tailed distributions using the Champernowne transformation Buch-Larsen, Nielsen, Guillen, Bolance, Kernel density estimation for heavy-tailed distributions using the
More information12 - Nonparametric Density Estimation
ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6
More informationPreface. 1 Nonparametric Density Estimation and Testing. 1.1 Introduction. 1.2 Univariate Density Estimation
Preface Nonparametric econometrics has become one of the most important sub-fields in modern econometrics. The primary goal of this lecture note is to introduce various nonparametric and semiparametric
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University
More informationTime Series and Forecasting Lecture 4 NonLinear Time Series
Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations
More informationLocal linear multiple regression with variable. bandwidth in the presence of heteroscedasticity
Local linear multiple regression with variable bandwidth in the presence of heteroscedasticity Azhong Ye 1 Rob J Hyndman 2 Zinai Li 3 23 January 2006 Abstract: We present local linear estimator with variable
More informationNonparametric Econometrics
Applied Microeconometrics with Stata Nonparametric Econometrics Spring Term 2011 1 / 37 Contents Introduction The histogram estimator The kernel density estimator Nonparametric regression estimators Semi-
More informationKernel Density Estimation
Kernel Density Estimation and Application in Discriminant Analysis Thomas Ledl Universität Wien Contents: Aspects of Application observations: 0 Which distribution? 0?? 0.0 0. 0. 0. 0.0 0. 0. 0 0 0.0
More informationKernel Density Estimation
Kernel Density Estimation Univariate Density Estimation Suppose tat we ave a random sample of data X 1,..., X n from an unknown continuous distribution wit probability density function (pdf) f(x) and cumulative
More informationA tour of kernel smoothing
Tarn Duong Institut Pasteur October 2007 The journey up till now 1995 1998 Bachelor, Univ. of Western Australia, Perth 1999 2000 Researcher, Australian Bureau of Statistics, Canberra and Sydney 2001 2004
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationIntegral approximation by kernel smoothing
Integral approximation by kernel smoothing François Portier Université catholique de Louvain - ISBA August, 29 2014 In collaboration with Bernard Delyon Topic of the talk: Given ϕ : R d R, estimation of
More informationMinimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model.
Minimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model By Michael Levine Purdue University Technical Report #14-03 Department of
More information4 Nonparametric Regression
4 Nonparametric Regression 4.1 Univariate Kernel Regression An important question in many fields of science is the relation between two variables, say X and Y. Regression analysis is concerned with the
More informationDensity and Distribution Estimation
Density and Distribution Estimation Nathaniel E. Helwig Assistant Professor of Psychology and Statistics University of Minnesota (Twin Cities) Updated 04-Jan-2017 Nathaniel E. Helwig (U of Minnesota) Density
More informationSmooth simultaneous confidence bands for cumulative distribution functions
Journal of Nonparametric Statistics, 2013 Vol. 25, No. 2, 395 407, http://dx.doi.org/10.1080/10485252.2012.759219 Smooth simultaneous confidence bands for cumulative distribution functions Jiangyan Wang
More informationNonparametric Methods
Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More informationNonparametric Function Estimation with Infinite-Order Kernels
Nonparametric Function Estimation with Infinite-Order Kernels Arthur Berg Department of Statistics, University of Florida March 15, 2008 Kernel Density Estimation (IID Case) Let X 1,..., X n iid density
More informationGoodness-of-fit tests for the cure rate in a mixture cure model
Biometrika (217), 13, 1, pp. 1 7 Printed in Great Britain Advance Access publication on 31 July 216 Goodness-of-fit tests for the cure rate in a mixture cure model BY U.U. MÜLLER Department of Statistics,
More informationA Least Squares Formulation for Canonical Correlation Analysis
A Least Squares Formulation for Canonical Correlation Analysis Liang Sun, Shuiwang Ji, and Jieping Ye Department of Computer Science and Engineering Arizona State University Motivation Canonical Correlation
More informationThe Gaussian distribution
The Gaussian distribution Probability density function: A continuous probability density function, px), satisfies the following properties:. The probability that x is between two points a and b b P a
More informationDensity estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas
0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity
More informationJ. Cwik and J. Koronacki. Institute of Computer Science, Polish Academy of Sciences. to appear in. Computational Statistics and Data Analysis
A Combined Adaptive-Mixtures/Plug-In Estimator of Multivariate Probability Densities 1 J. Cwik and J. Koronacki Institute of Computer Science, Polish Academy of Sciences Ordona 21, 01-237 Warsaw, Poland
More informationAn Introduction to Multivariate Statistical Analysis
An Introduction to Multivariate Statistical Analysis Third Edition T. W. ANDERSON Stanford University Department of Statistics Stanford, CA WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Contents
More informationLocal linear multivariate. regression with variable. bandwidth in the presence of. heteroscedasticity
Model ISSN 1440-771X Department of Econometrics and Business Statistics http://www.buseco.monash.edu.au/depts/ebs/pubs/wpapers/ Local linear multivariate regression with variable bandwidth in the presence
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationMotivational Example
Motivational Example Data: Observational longitudinal study of obesity from birth to adulthood. Overall Goal: Build age-, gender-, height-specific growth charts (under 3 year) to diagnose growth abnomalities.
More informationNonparametric Regression. Changliang Zou
Nonparametric Regression Institute of Statistics, Nankai University Email: nk.chlzou@gmail.com Smoothing parameter selection An overall measure of how well m h (x) performs in estimating m(x) over x (0,
More informationLog-Density Estimation with Application to Approximate Likelihood Inference
Log-Density Estimation with Application to Approximate Likelihood Inference Martin Hazelton 1 Institute of Fundamental Sciences Massey University 19 November 2015 1 Email: m.hazelton@massey.ac.nz WWPMS,
More informationLocal Polynomial Regression
VI Local Polynomial Regression (1) Global polynomial regression We observe random pairs (X 1, Y 1 ),, (X n, Y n ) where (X 1, Y 1 ),, (X n, Y n ) iid (X, Y ). We want to estimate m(x) = E(Y X = x) based
More informationCross-fitting and fast remainder rates for semiparametric estimation
Cross-fitting and fast remainder rates for semiparametric estimation Whitney K. Newey James M. Robins The Institute for Fiscal Studies Department of Economics, UCL cemmap working paper CWP41/17 Cross-Fitting
More informationON SOME TWO-STEP DENSITY ESTIMATION METHOD
UNIVESITATIS IAGELLONICAE ACTA MATHEMATICA, FASCICULUS XLIII 2005 ON SOME TWO-STEP DENSITY ESTIMATION METHOD by Jolanta Jarnicka Abstract. We introduce a new two-step kernel density estimation method,
More informationAdaptive Nonparametric Density Estimators
Adaptive Nonparametric Density Estimators by Alan J. Izenman Introduction Theoretical results and practical application of histograms as density estimators usually assume a fixed-partition approach, where
More informationNonparametric Density Estimation. October 1, 2018
Nonparametric Density Estimation October 1, 2018 Introduction If we can t fit a distribution to our data, then we use nonparametric density estimation. Start with a histogram. But there are problems with
More informationVariance Function Estimation in Multivariate Nonparametric Regression
Variance Function Estimation in Multivariate Nonparametric Regression T. Tony Cai 1, Michael Levine Lie Wang 1 Abstract Variance function estimation in multivariate nonparametric regression is considered
More informationSmoothness Adaptive Average Derivative Estimation
Econometrics Journal 009, volume 10, pp. 1 3. Article No. ectj?????? Smoothness Adaptive Average Derivative Estimation Marcia M.A. Schafgans and Victoria Zinde-Walsh Department of Economics, London School
More informationNotes on Random Vectors and Multivariate Normal
MATH 590 Spring 06 Notes on Random Vectors and Multivariate Normal Properties of Random Vectors If X,, X n are random variables, then X = X,, X n ) is a random vector, with the cumulative distribution
More informationMultivariate Locally Weighted Polynomial Fitting and Partial Derivative Estimation
journal of multivariate analysis 59, 8705 (996) article no. 0060 Multivariate Locally Weighted Polynomial Fitting and Partial Derivative Estimation Zhan-Qian Lu Geophysical Statistics Project, National
More informationKullback-Leibler Designs
Kullback-Leibler Designs Astrid JOURDAN Jessica FRANCO Contents Contents Introduction Kullback-Leibler divergence Estimation by a Monte-Carlo method Design comparison Conclusion 2 Introduction Computer
More informationApplications of nonparametric methods in economic and political science
Applications of nonparametric methods in economic and political science Dissertation presented for the degree of Doctor rerum politicarum at the Faculty of Economic Sciences of the Georg-August-Universität
More informationRank Estimation of Partially Linear Index Models
Rank Estimation of Partially Linear Index Models Jason Abrevaya University of Texas at Austin Youngki Shin University of Western Ontario October 2008 Preliminary Do not distribute Abstract We consider
More informationUC Berkeley Department of Electrical Engineering and Computer Sciences. EECS 126: Probability and Random Processes
UC Berkeley Department of Electrical Engineering and Computer Sciences EECS 6: Probability and Random Processes Problem Set 3 Spring 9 Self-Graded Scores Due: February 8, 9 Submit your self-graded scores
More informationO Combining cross-validation and plug-in methods - for kernel density bandwidth selection O
O Combining cross-validation and plug-in methods - for kernel density selection O Carlos Tenreiro CMUC and DMUC, University of Coimbra PhD Program UC UP February 18, 2011 1 Overview The nonparametric problem
More informationStatistics for Python
Statistics for Python An extension module for the Python scripting language Michiel de Hoon, Columbia University 2 September 2010 Statistics for Python, an extension module for the Python scripting language.
More informationDiscrete Mathematics and Probability Theory Fall 2015 Lecture 21
CS 70 Discrete Mathematics and Probability Theory Fall 205 Lecture 2 Inference In this note we revisit the problem of inference: Given some data or observations from the world, what can we infer about
More informationDEPARTMENT MATHEMATIK ARBEITSBEREICH MATHEMATISCHE STATISTIK UND STOCHASTISCHE PROZESSE
Estimating the error distribution in nonparametric multiple regression with applications to model testing Natalie Neumeyer & Ingrid Van Keilegom Preprint No. 2008-01 July 2008 DEPARTMENT MATHEMATIK ARBEITSBEREICH
More informationTitolo Smooth Backfitting with R
Rapporto n. 176 Titolo Smooth Backfitting with R Autori Alberto Arcagni, Luca Bagnato ottobre 2009 Dipartimento di Metodi Quantitativi per le Scienze Economiche ed Aziendali Università degli Studi di Milano
More information3 Nonparametric Density Estimation
3 Nonparametric Density Estimation Example: Income distribution Source: U.K. Family Expenditure Survey (FES) 1968-1995 Approximately 7000 British Households per year For each household many different variables
More informationA Goodness-of-fit Test for Copulas
A Goodness-of-fit Test for Copulas Artem Prokhorov August 2008 Abstract A new goodness-of-fit test for copulas is proposed. It is based on restrictions on certain elements of the information matrix and
More informationECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd
ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation Petra E. Todd Fall, 2014 2 Contents 1 Review of Stochastic Order Symbols 1 2 Nonparametric Density Estimation 3 2.1 Histogram
More informationNonparametric Inference via Bootstrapping the Debiased Estimator
Nonparametric Inference via Bootstrapping the Debiased Estimator Yen-Chi Chen Department of Statistics, University of Washington ICSA-Canada Chapter Symposium 2017 1 / 21 Problem Setup Let X 1,, X n be
More informationA Probability Review
A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in
More informationOptimal Bandwidth Choice for the Regression Discontinuity Estimator
Optimal Bandwidth Choice for the Regression Discontinuity Estimator Guido Imbens and Karthik Kalyanaraman First Draft: June 8 This Draft: September Abstract We investigate the choice of the bandwidth for
More informationFunctional Latent Feature Models. With Single-Index Interaction
Generalized With Single-Index Interaction Department of Statistics Center for Statistical Bioinformatics Institute for Applied Mathematics and Computational Science Texas A&M University Naisyin Wang and
More informationESTIMATORS IN THE CONTEXT OF ACTUARIAL LOSS MODEL A COMPARISON OF TWO NONPARAMETRIC DENSITY MENGJUE TANG A THESIS MATHEMATICS AND STATISTICS
A COMPARISON OF TWO NONPARAMETRIC DENSITY ESTIMATORS IN THE CONTEXT OF ACTUARIAL LOSS MODEL MENGJUE TANG A THESIS IN THE DEPARTMENT OF MATHEMATICS AND STATISTICS PRESENTED IN PARTIAL FULFILLMENT OF THE
More informationLog-transform kernel density estimation of income distribution
Log-transform kernel density estimation of income distribution by Arthur Charpentier Université du Québec à Montréal CREM & GERAD Département de Mathématiques 201, av. Président-Kennedy, PK-5151, Montréal,
More informationDensity Estimation (II)
Density Estimation (II) Yesterday Overview & Issues Histogram Kernel estimators Ideogram Today Further development of optimization Estimating variance and bias Adaptive kernels Multivariate kernel estimation
More informationPREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY
REVSTAT Statistical Journal Volume 7, Number 1, April 2009, 37 54 PREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY Authors: Germán Aneiros-Pérez Departamento de Matemáticas,
More informationBandwidth Selection in Nonparametric Kernel Estimation
Bandwidth Selection in Nonparametric Kernel Estimation Dissertation presented for the degree of Doctor rerum politicarum at the Faculty of Economic Sciences of the Georg-August-Universität Göttingen by
More informationA PROBABILITY DENSITY FUNCTION ESTIMATION USING F-TRANSFORM
K Y BERNETIKA VOLUM E 46 ( 2010), NUMBER 3, P AGES 447 458 A PROBABILITY DENSITY FUNCTION ESTIMATION USING F-TRANSFORM Michal Holčapek and Tomaš Tichý The aim of this paper is to propose a new approach
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationComputational Methods. Least Squares Approximation/Optimization
Computational Methods Least Squares Approximation/Optimization Manfred Huber 2011 1 Least Squares Least squares methods are aimed at finding approximate solutions when no precise solution exists Find the
More informationAcceleration of some empirical means. Application to semiparametric regression
Acceleration of some empirical means. Application to semiparametric regression François Portier Université catholique de Louvain - ISBA November, 8 2013 In collaboration with Bernard Delyon Regression
More informationSUPPLEMENTARY MATERIAL FOR PUBLICATION ONLINE 1
SUPPLEMENTARY MATERIAL FOR PUBLICATION ONLINE 1 B Technical details B.1 Variance of ˆf dec in the ersmooth case We derive the variance in the case where ϕ K (t) = ( 1 t 2) κ I( 1 t 1), i.e. we take K as
More informationPARSIMONIOUS MULTIVARIATE COPULA MODEL FOR DENSITY ESTIMATION. Alireza Bayestehtashk and Izhak Shafran
PARSIMONIOUS MULTIVARIATE COPULA MODEL FOR DENSITY ESTIMATION Alireza Bayestehtashk and Izhak Shafran Center for Spoken Language Understanding, Oregon Health & Science University, Portland, Oregon, USA
More informationMinimum Error Rate Classification
Minimum Error Rate Classification Dr. K.Vijayarekha Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur-613 401 Table of Contents 1.Minimum Error Rate Classification...
More informationOptimal bandwidth selection for differences of nonparametric estimators with an application to the sharp regression discontinuity design
Optimal bandwidth selection for differences of nonparametric estimators with an application to the sharp regression discontinuity design Yoichi Arai Hidehiko Ichimura The Institute for Fiscal Studies Department
More informationMinimum Hellinger Distance Estimation in a. Semiparametric Mixture Model
Minimum Hellinger Distance Estimation in a Semiparametric Mixture Model Sijia Xiang 1, Weixin Yao 1, and Jingjing Wu 2 1 Department of Statistics, Kansas State University, Manhattan, Kansas, USA 66506-0802.
More informationRegression #5: Confidence Intervals and Hypothesis Testing (Part 1)
Regression #5: Confidence Intervals and Hypothesis Testing (Part 1) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #5 1 / 24 Introduction What is a confidence interval? To fix ideas, suppose
More informationAkaike criterion: Kullback-Leibler discrepancy
Model choice. Akaike s criterion Akaike criterion: Kullback-Leibler discrepancy Given a family of probability densities {f ( ; ψ), ψ Ψ}, Kullback-Leibler s index of f ( ; ψ) relative to f ( ; θ) is (ψ
More informationprinting Three areas: solid calculus, particularly calculus of several
Math 5610 printing 5600 5610 Notes of 8/21/18 Quick Review of some Prerequisites Three areas: solid calculus, particularly calculus of several variables. linear algebra Programming (Coding) The term project
More informationError distribution function for parametrically truncated and censored data
Error distribution function for parametrically truncated and censored data Géraldine LAURENT Jointly with Cédric HEUCHENNE QuantOM, HEC-ULg Management School - University of Liège Friday, 14 September
More informationLOCAL LINEAR REGRESSION FOR GENERALIZED LINEAR MODELS WITH MISSING DATA
The Annals of Statistics 1998, Vol. 26, No. 3, 1028 1050 LOCAL LINEAR REGRESSION FOR GENERALIZED LINEAR MODELS WITH MISSING DATA By C. Y. Wang, 1 Suojin Wang, 2 Roberto G. Gutierrez and R. J. Carroll 3
More information10-701/ Machine Learning - Midterm Exam, Fall 2010
10-701/15-781 Machine Learning - Midterm Exam, Fall 2010 Aarti Singh Carnegie Mellon University 1. Personal info: Name: Andrew account: E-mail address: 2. There should be 15 numbered pages in this exam
More informationIntroduction to Regression
Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric
More informationStatistics 910, #5 1. Regression Methods
Statistics 910, #5 1 Overview Regression Methods 1. Idea: effects of dependence 2. Examples of estimation (in R) 3. Review of regression 4. Comparisons and relative efficiencies Idea Decomposition Well-known
More informationBasic Nonparametric Estimation Spring 2002
Basic Nonparametric Estimation Spring 2002 Te following topics are covered today: Basic Nonparametric Regression. Tere are four books tat you can find reference: Silverman986, Wand and Jones995, Hardle990,
More informationECONOMETRICS II (ECO 2401S) University of Toronto. Department of Economics. Spring 2013 Instructor: Victor Aguirregabiria
ECONOMETRICS II (ECO 2401S) University of Toronto. Department of Economics. Spring 2013 Instructor: Victor Aguirregabiria SOLUTION TO FINAL EXAM Friday, April 12, 2013. From 9:00-12:00 (3 hours) INSTRUCTIONS:
More informationThe Gerber Statistic: A Robust Measure of Correlation
The Gerber Statistic: A Robust Measure of Correlation Sander Gerber Babak Javid Harry Markowitz Paul Sargen David Starer February 21, 2019 Abstract We introduce the Gerber statistic, a robust measure of
More informationA COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES
A COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES MICHAEL O HARA AND CHRISTOPHER F. PARMETER Abstract. This paper presents a Monte Carlo comparison of
More informationWavelet Regression Estimation in Longitudinal Data Analysis
Wavelet Regression Estimation in Longitudinal Data Analysis ALWELL J. OYET and BRAJENDRA SUTRADHAR Department of Mathematics and Statistics, Memorial University of Newfoundland St. John s, NF Canada, A1C
More informationNonparametric Regression 10/ Larry Wasserman
Nonparametric Regression 10/36-702 Larry Wasserman 1 Introduction Now we focus on the following problem: Given a sample (X 1, Y 1 ),..., (X n, Y n ), where X i R d and Y i R, estimate the regression function
More informationSupplementary Materials to Convex Banding of the Covariance Matrix
Supplementary Materials to Convex Banding of the Covariance Matrix Jacob Bien, Florentina Bunea, Luo Xiao May 25, 2015 A.1 Dual problem Define LΣ, A 1,..., A = 1 2 S Σ 2 F + λ W l A l, Σ. Observe that
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression
More informationAkaike criterion: Kullback-Leibler discrepancy
Model choice. Akaike s criterion Akaike criterion: Kullback-Leibler discrepancy Given a family of probability densities {f ( ; ), 2 }, Kullback-Leibler s index of f ( ; ) relativetof ( ; ) is Z ( ) =E
More informationOn robust and efficient estimation of the center of. Symmetry.
On robust and efficient estimation of the center of symmetry Howard D. Bondell Department of Statistics, North Carolina State University Raleigh, NC 27695-8203, U.S.A (email: bondell@stat.ncsu.edu) Abstract
More informationStatistica Sinica Preprint No: SS
Statistica Sinica Preprint No: SS-017-0013 Title A Bootstrap Method for Constructing Pointwise and Uniform Confidence Bands for Conditional Quantile Functions Manuscript ID SS-017-0013 URL http://wwwstatsinicaedutw/statistica/
More informationIntroduction to Nonparametric and Semiparametric Estimation. Good when there are lots of data and very little prior information on functional form.
1 Introduction to Nonparametric and Semiparametric Estimation Good when there are lots of data and very little prior information on functional form. Examples: y = f(x) + " (nonparametric) y = z 0 + f(x)
More informationFunction of Longitudinal Data
New Local Estimation Procedure for Nonparametric Regression Function of Longitudinal Data Weixin Yao and Runze Li Abstract This paper develops a new estimation of nonparametric regression functions for
More informationBootstrap, Jackknife and other resampling methods
Bootstrap, Jackknife and other resampling methods Part III: Parametric Bootstrap Rozenn Dahyot Room 128, Department of Statistics Trinity College Dublin, Ireland dahyot@mee.tcd.ie 2005 R. Dahyot (TCD)
More informationA Monte Carlo Investigation of Smoothing Methods. for Error Density Estimation in Functional Data. Analysis with an Illustrative Application to a
A Monte Carlo Investigation of Smoothing Methods for Error Density Estimation in Functional Data Analysis with an Illustrative Application to a Chemometric Data Set A MONTE CARLO INVESTIGATION OF SMOOTHING
More informationLogistic Kernel Estimator and Bandwidth Selection. for Density Function
International Journal of Contemporary Matematical Sciences Vol. 13, 2018, no. 6, 279-286 HIKARI Ltd, www.m-ikari.com ttps://doi.org/10.12988/ijcms.2018.81133 Logistic Kernel Estimator and Bandwidt Selection
More information