Spatially Smoothed Kernel Density Estimation via Generalized Empirical Likelihood

Size: px
Start display at page:

Download "Spatially Smoothed Kernel Density Estimation via Generalized Empirical Likelihood"

Transcription

1 Spatially Smoothed Kernel Density Estimation via Generalized Empirical Likelihood Kuangyu Wen & Ximing Wu Texas A&M University Info-Metrics Institute Conference: Recent Innovations in Info-Metrics October 2014 Spatial Kernel Density Estimation 1 / 20

2 Motivation Nonparametric density estimation offers a flexible alternative to parametric method. To obtain the same level of precision as that of its parametric counterpart, a larger sample size is desired in nonparametric estimation due to its slower convergence rate. Given similarity/dependence among distributions across geographic units, efficiency gain is possible via information pooling; particularly so in nonparametric estimations with small sample size for each unit. Example: Crop yield distributions are typically estimated (individually) at the county level. Taking into account their spatial dependence may improve estimation. Spatial Kernel Density Estimation 2 / 20

3 Nonparametric estimation of spatially dependent densities An information-theoretic nonparametric estimation procedure Weighted Kernel Density Estimator (WKDE) that satisfies moment conditions implied by spatial dependence among units Observation weights are the implied Generalized Empirical Likelihood (GEL) probabilities associated with spatial moment conditions. GEL offers a flexible yet simple framework to incorporate out-of-sample information into density estimation. Computationally inexpensive; joint estimation of a large number of spatially dependent densities is avoided. Spatial Kernel Density Estimation 3 / 20

4 Talk plan Weighted KDE (WKDE) via the GEL Spatial smoothing of KDE Monte Carlo Simulations Empirical example Spatial Kernel Density Estimation 4 / 20

5 Kernel Density Estimation (KDE) Data: X 1,..., X T i.i.d. from a continuously differentiable distribution F with density f KDE: ˆf (x) = 1 Th ( ) x Xt K 1 h T K h (x X t ) where K is a kernel function and h the bandwidth. Weighted KDE (WKDE) ˆf (x) = p t K h (x X t ) where p = (p 1,..., p T ) are non-negative observation weights that sum to one. Spatial Kernel Density Estimation 5 / 20

6 Incorporation of out-of-sample information Incorporation of out-of-sample information, for instance population moments or theoretical properties, may enhance the KDE. Suppose out-of-sample information comes in the form E[g m (x)] = µ m, m = 1,..., M, where g m are known real functions. Chen (1997) uses WKDE to incorporate out-of-sample information; the weights is calculated via empirical likelihood (Owen, 1988; 2001) max p 1,...,p T s.t. log p t p t g m (X t ) = µ m, m = 1,..., M, p t 0, p t = 1. Spatial Kernel Density Estimation 6 / 20

7 Empirical likelihood weighted KDE: ˆf EL (x) = ˆp t K h (x X t ) where ˆp = (ˆp 1,..., ˆp T ) are the implied EL probabilities. Under mild regularity conditions abias(ˆf EL (x)) = abias(ˆf (x)) + o(n 1 ) avar(ˆf EL (x)) = avar(ˆf (x)) r(x)f 2 (x)/n + o(n 1 ) where r(x) : R R + depends on the unknown density f and moment functions g = (g 1,..., g M ). An important implication: ˆf EL can use the optimal bandwidth for ˆf ; joint search of p and bandwidth is not required. Spatial Kernel Density Estimation 7 / 20

8 Extension to Generalized Empirical Likelihood Cressie-Read power discrepancy CR ν = 1 ν(ν + 1) ( (npt ) ν 1 ) Generalized Empirical Likelihood (GEL) 1 ( min (npt ) ν 1 ) p 1,...,p T ν(ν + 1) s.t. p t g m (X t ) = µ m, m = 1,..., M, p t 0, p t = 1. ν = 0 = EL ν = 1 = Exponential Tilting (ET) or maximum entropy Spatial Kernel Density Estimation 8 / 20

9 Spatial smoothing of KDE Consider N spatially dependent densities f i, i = 1,..., N; for each i, we observe Y i = (Y i,1,..., Y i,t ) i.i.d. from f i. Given selected moment functions g = (g 1,..., g M ), define their sample analog for each unit ĝ (i) = 1 T g(y i,t ), i = 1,..., N. Spatial weights for unit i: w i = {w i,j } N j=1 ; w i,j 0 and N j=1 w i,j = 1. Spatially smoothed sample moments g (i) = ( N 1 w i,j T j=1 ) g(y j,t ) = N w i,j ĝ (j) j=1 Spatial Kernel Density Estimation 9 / 20

10 Estimate f i using the WKDE ˆf EL or ˆf ET subject to spatial moment conditions g (i), i = 1,..., N. For implication, we need to specify spatial weights w = {w 1,..., w N } moment functions g = (g 1,..., g M ) Spatial Kernel Density Estimation 10 / 20

11 Construction of spatial weights d i,j 0: the spatial distance between unit i and j N k i : an indicator set including unit i and its k nearest neighbors Spatial weights for unit i: w i,j = exp( di,j 2 /γ i)1(j Ni k ) N j=1 exp( d i,j 2 /γ, j = 1,..., N, i)1(j Ni k ) where 1 is the indicator function and γ i = 1 k j N k i d i,j Spatial Kernel Density Estimation 11 / 20

12 Specification of moment functions Consider for now the Exponential Tilting estimator ˆf ET. It can be shown to be asymptotically equivalent to a minimum cross entropy density estimator with the KDE ˆf as the reference density: f ET = arg min s.t. f f (x)dx = 1, The solution takes the form { f ET (x) = ˆf (x) exp f (x) log f (x) ˆf (x) dx g m (x)f (x)dx = µ m, m = 1,..., M. λ 0 + } M λ m g m (x) = ˆf ET (x) + o p (n 1 ) m=1 where λ 0 is a normalization constant. Spatial Kernel Density Estimation 12 / 20

13 Thus ˆf ET is seen to modify ˆf with a multiplicative adjustment, which consists of a basis expansion in exponent. Therefore the selection of moment functions for ˆf ET amounts to the selection of basis functions for nonparametric jminimum cross entropy density estimation. Power series, g(x) = (x, x 2, x 3,...), is customarily used, but can be sensitive to outliers and require the existence of population moments up to order M. We use spline basis due to its flexibility and robustness.; e.g., g(x) = (x, x 2, (x K 1 ) 2 +, (x K 2 ) 2 +,..., (x K M ) 2 +), where (x) + = max(0, x) and K 1,..., K M are spline knots. Only the first two moments are required to exist. The resultant density can be viewed as a hybrid of KDE and log spline estimator (Kooperberg and Stone, 1992; Stone, et al., 1997) Spatial Kernel Density Estimation 13 / 20

14 Monte Carlo Simulations N = 100 locations in a unit square I Construct a grid evenly spaced over I Randomly draw 100 points from the grid without replacement Calculate the distance matrix D = [d i,j ] Spatial Kernel Density Estimation 14 / 20

15 Constructions of spatially dependent distributions Assume f i Skewed Normal (µ i, σ i, α i ) for i = 1,, N. µ = (µ 1,, µ N ), σ = (σ 1,, σ N ) and α = (α 1,, α N ). A N-dimensional vector θ follows spatial error model SpE (β, λ, σ ɛ ) if θ = β + η η = λwη + ɛ, where ɛ N ( 0, σɛ 2 ) I N and the spatial weighting matrix W is constructed as 1 W = [ ] wi,j, w i,j = 1 if i j and the location i is one of the 5 nearest neighbors of the location j; otherwise wi,j = 0; 2 row standardization, i.e. W = [w i,j ], w i,j = wi,j / N j=1 w i,j. Spatial Kernel Density Estimation 15 / 20

16 We set µ SpE ( 1, 0.8, 0.03), log (σ) SpE (0, 0.8, 0.03) and α SpE ( 0.75, 0.8, 0.03). Each vector of shape parameter is given by θ = β + (I N λw) 1 ɛ. Randomly draw ɛ to obtain a set of values for each µ, σ and α respectively Spatial Kernel Density Estimation 16 / 20

17 Generate data Y i = {Y i,t } T from f i, i = 1,..., N Sample size T = 30 and 60 We consider the KDE ˆf and two WKDEs: ˆf EL and ˆf ET. For each unit, KDE bandwidth is selected according to the rule of thumb. Moment functions: quadratic splines with two knots at the 33th and 66th percentiles of the pooled sample: g(y) = (y, y 2, (y Q 33 ) 2 +, (y Q 66 ) 2 +), where (y) + = max(y, 0) and Q α is the αth sample percentile. Number of nearest neighbors in the calculation of spatial weights: k = 5 Number of repetitions: R = 500 Spatial Kernel Density Estimation 17 / 20

18 Second experiment Construct f i as a two-component normal mixture, i.e. f i = ω i φ(µ 1,i, σ 1,i ) + (1 ω i )φ(µ 2,i, σ 2,i ). Parameters: Φ 1 (ω) SpE(Φ 1 (0.2), 0.6, 0.03), µ 1 SpE( 3, 0.6, 0.2), µ 2 SpE(0, 0.6, 0.1), log (σ 1 ) SpE(log(1.5), 0.6, 0.03) and log (σ 2 ) SpE(0, 0.6, 0.03) Spatial Kernel Density Estimation 18 / 20

19 Evaluation of performance ˆf (r) Denote by i an estimate of f i in the rth experiment, i = 1,..., N and r = 1,..., R. Global performance: average mean integrated squared error 1 NR N R i=1 r=1 (ˆf (r) i f i ) 2 Tail performance: let Q i,5 be the 5% quantile of the f i and ˆπ (r) i = x Q i,5 ˆf (r) i (x)dx, the average mean squared error of tail probability estimation 1 NR N R i=1 r=1 ( ) ˆπ (r) 2 i 5% Spatial Kernel Density Estimation 19 / 20

20 Simulation results MSE of KDE and ratio of WKDE MSE relative to that of KDE KDE WKDE EL ET T = 30 SN global % 63.89% tail % 56.50% NM global % 78.21% tail % 34.90% T = 60 SN global % 65.05% tail % 63.53% NM global % 76.74% tail % 38.11% Spatial Kernel Density Estimation 20 / 20

Motivational Example

Motivational Example Motivational Example Data: Observational longitudinal study of obesity from birth to adulthood. Overall Goal: Build age-, gender-, height-specific growth charts (under 3 year) to diagnose growth abnomalities.

More information

Transformation-based Nonparametric Estimation of Multivariate Densities

Transformation-based Nonparametric Estimation of Multivariate Densities Transformation-based Nonparametric Estimation of Multivariate Densities Meng-Shiuh Chang Ximing Wu March 9, 2013 Abstract We propose a probability-integral-transformation-based estimator of multivariate

More information

12 - Nonparametric Density Estimation

12 - Nonparametric Density Estimation ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6

More information

GMM Estimation of a Maximum Entropy Distribution with Interval Data

GMM Estimation of a Maximum Entropy Distribution with Interval Data GMM Estimation of a Maximum Entropy Distribution with Interval Data Ximing Wu and Jeffrey M. Perloff January, 2005 Abstract We develop a GMM estimator for the distribution of a variable where summary statistics

More information

Teruko Takada Department of Economics, University of Illinois. Abstract

Teruko Takada Department of Economics, University of Illinois. Abstract Nonparametric density estimation: A comparative study Teruko Takada Department of Economics, University of Illinois Abstract Motivated by finance applications, the objective of this paper is to assess

More information

STAT 518 Intro Student Presentation

STAT 518 Intro Student Presentation STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible

More information

Minimum Hellinger Distance Estimation in a. Semiparametric Mixture Model

Minimum Hellinger Distance Estimation in a. Semiparametric Mixture Model Minimum Hellinger Distance Estimation in a Semiparametric Mixture Model Sijia Xiang 1, Weixin Yao 1, and Jingjing Wu 2 1 Department of Statistics, Kansas State University, Manhattan, Kansas, USA 66506-0802.

More information

Bayesian estimation of the discrepancy with misspecified parametric models

Bayesian estimation of the discrepancy with misspecified parametric models Bayesian estimation of the discrepancy with misspecified parametric models Pierpaolo De Blasi University of Torino & Collegio Carlo Alberto Bayesian Nonparametrics workshop ICERM, 17-21 September 2012

More information

Statistics: Learning models from data

Statistics: Learning models from data DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial

More information

Statistical Properties of Numerical Derivatives

Statistical Properties of Numerical Derivatives Statistical Properties of Numerical Derivatives Han Hong, Aprajit Mahajan, and Denis Nekipelov Stanford University and UC Berkeley November 2010 1 / 63 Motivation Introduction Many models have objective

More information

Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics. Jiti Gao

Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics. Jiti Gao Model Specification Testing in Nonparametric and Semiparametric Time Series Econometrics Jiti Gao Department of Statistics School of Mathematics and Statistics The University of Western Australia Crawley

More information

Nonparametric Tests for Multi-parameter M-estimators

Nonparametric Tests for Multi-parameter M-estimators Nonparametric Tests for Multi-parameter M-estimators John Robinson School of Mathematics and Statistics University of Sydney The talk is based on joint work with John Kolassa. It follows from work over

More information

Lecture 3: Statistical Decision Theory (Part II)

Lecture 3: Statistical Decision Theory (Part II) Lecture 3: Statistical Decision Theory (Part II) Hao Helen Zhang Hao Helen Zhang Lecture 3: Statistical Decision Theory (Part II) 1 / 27 Outline of This Note Part I: Statistics Decision Theory (Classical

More information

Can we do statistical inference in a non-asymptotic way? 1

Can we do statistical inference in a non-asymptotic way? 1 Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.

More information

Lecture 13: Subsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf

Lecture 13: Subsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf Lecture 13: 2011 Bootstrap ) R n x n, θ P)) = τ n ˆθn θ P) Example: ˆθn = X n, τ n = n, θ = EX = µ P) ˆθ = min X n, τ n = n, θ P) = sup{x : F x) 0} ) Define: J n P), the distribution of τ n ˆθ n θ P) under

More information

Quantifying Stochastic Model Errors via Robust Optimization

Quantifying Stochastic Model Errors via Robust Optimization Quantifying Stochastic Model Errors via Robust Optimization IPAM Workshop on Uncertainty Quantification for Multiscale Stochastic Systems and Applications Jan 19, 2016 Henry Lam Industrial & Operations

More information

Confidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods

Confidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods Chapter 4 Confidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods 4.1 Introduction It is now explicable that ridge regression estimator (here we take ordinary ridge estimator (ORE)

More information

41903: Introduction to Nonparametrics

41903: Introduction to Nonparametrics 41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific

More information

AN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY

AN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY Econometrics Working Paper EWP0401 ISSN 1485-6441 Department of Economics AN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY Lauren Bin Dong & David E. A. Giles Department of Economics, University of Victoria

More information

Information Geometry

Information Geometry 2015 Workshop on High-Dimensional Statistical Analysis Dec.11 (Friday) ~15 (Tuesday) Humanities and Social Sciences Center, Academia Sinica, Taiwan Information Geometry and Spontaneous Data Learning Shinto

More information

A Shape Constrained Estimator of Bidding Function of First-Price Sealed-Bid Auctions

A Shape Constrained Estimator of Bidding Function of First-Price Sealed-Bid Auctions A Shape Constrained Estimator of Bidding Function of First-Price Sealed-Bid Auctions Yu Yvette Zhang Abstract This paper is concerned with economic analysis of first-price sealed-bid auctions with risk

More information

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite

More information

Econometrics I, Estimation

Econometrics I, Estimation Econometrics I, Estimation Department of Economics Stanford University September, 2008 Part I Parameter, Estimator, Estimate A parametric is a feature of the population. An estimator is a function of the

More information

Time Series and Forecasting Lecture 4 NonLinear Time Series

Time Series and Forecasting Lecture 4 NonLinear Time Series Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations

More information

Additive Isotonic Regression

Additive Isotonic Regression Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive

More information

Quantile Regression for Extraordinarily Large Data

Quantile Regression for Extraordinarily Large Data Quantile Regression for Extraordinarily Large Data Shih-Kang Chao Department of Statistics Purdue University November, 2016 A joint work with Stanislav Volgushev and Guang Cheng Quantile regression Two-step

More information

Model-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego

Model-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego Model-free prediction intervals for regression and autoregression Dimitris N. Politis University of California, San Diego To explain or to predict? Models are indispensable for exploring/utilizing relationships

More information

Analysis methods of heavy-tailed data

Analysis methods of heavy-tailed data Institute of Control Sciences Russian Academy of Sciences, Moscow, Russia February, 13-18, 2006, Bamberg, Germany June, 19-23, 2006, Brest, France May, 14-19, 2007, Trondheim, Norway PhD course Chapter

More information

STAT 512 sp 2018 Summary Sheet

STAT 512 sp 2018 Summary Sheet STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}

More information

Empirical Likelihood Inference for Two-Sample Problems

Empirical Likelihood Inference for Two-Sample Problems Empirical Likelihood Inference for Two-Sample Problems by Ying Yan A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Master of Mathematics in Statistics

More information

Bayesian Aggregation for Extraordinarily Large Dataset

Bayesian Aggregation for Extraordinarily Large Dataset Bayesian Aggregation for Extraordinarily Large Dataset Guang Cheng 1 Department of Statistics Purdue University www.science.purdue.edu/bigdata Department Seminar Statistics@LSE May 19, 2017 1 A Joint Work

More information

Bickel Rosenblatt test

Bickel Rosenblatt test University of Latvia 28.05.2011. A classical Let X 1,..., X n be i.i.d. random variables with a continuous probability density function f. Consider a simple hypothesis H 0 : f = f 0 with a significance

More information

Single Index Quantile Regression for Heteroscedastic Data

Single Index Quantile Regression for Heteroscedastic Data Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University JSM, 2015 E. Christou, M. G. Akritas (PSU) SIQR JSM, 2015

More information

A Recursive Formula for the Kaplan-Meier Estimator with Mean Constraints

A Recursive Formula for the Kaplan-Meier Estimator with Mean Constraints Noname manuscript No. (will be inserted by the editor) A Recursive Formula for the Kaplan-Meier Estimator with Mean Constraints Mai Zhou Yifan Yang Received: date / Accepted: date Abstract In this note

More information

GMM Estimation of a Maximum Entropy Distribution with Interval Data

GMM Estimation of a Maximum Entropy Distribution with Interval Data GMM Estimation of a Maximum Entropy Distribution with Interval Data Ximing Wu and Jeffrey M. Perloff March 2005 Abstract We develop a GMM estimator for the distribution of a variable where summary statistics

More information

Introduction to Regression

Introduction to Regression Introduction to Regression David E Jones (slides mostly by Chad M Schafer) June 1, 2016 1 / 102 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures

More information

Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures

Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures David Hunter Pennsylvania State University, USA Joint work with: Tom Hettmansperger, Hoben Thomas, Didier Chauveau, Pierre Vandekerkhove,

More information

Single Index Quantile Regression for Heteroscedastic Data

Single Index Quantile Regression for Heteroscedastic Data Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University SMAC, November 6, 2015 E. Christou, M. G. Akritas (PSU) SIQR

More information

Introduction to Regression

Introduction to Regression Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric

More information

Density estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas

Density estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas 0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity

More information

Introduction to Empirical Processes and Semiparametric Inference Lecture 02: Overview Continued

Introduction to Empirical Processes and Semiparametric Inference Lecture 02: Overview Continued Introduction to Empirical Processes and Semiparametric Inference Lecture 02: Overview Continued Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics and Operations Research

More information

Sliced Inverse Regression

Sliced Inverse Regression Sliced Inverse Regression Ge Zhao gzz13@psu.edu Department of Statistics The Pennsylvania State University Outline Background of Sliced Inverse Regression (SIR) Dimension Reduction Definition of SIR Inversed

More information

Likelihood Inference for Lattice Spatial Processes

Likelihood Inference for Lattice Spatial Processes Likelihood Inference for Lattice Spatial Processes Donghoh Kim November 30, 2004 Donghoh Kim 1/24 Go to 1234567891011121314151617 FULL Lattice Processes Model : The Ising Model (1925), The Potts Model

More information

Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

Statistics - Lecture One. Outline. Charlotte Wickham  1. Basic ideas about estimation Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence

More information

Asymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands

Asymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands Asymptotic Multivariate Kriging Using Estimated Parameters with Bayesian Prediction Methods for Non-linear Predictands Elizabeth C. Mannshardt-Shamseldin Advisor: Richard L. Smith Duke University Department

More information

Method of Moments. which we usually denote by X or sometimes by X n to emphasize that there are n observations.

Method of Moments. which we usually denote by X or sometimes by X n to emphasize that there are n observations. Method of Moments Definition. If {X 1,..., X n } is a sample from a population, then the empirical k-th moment of this sample is defined to be X k 1 + + Xk n n Example. For a sample {X 1, X, X 3 } the

More information

Kernel density estimation for heavy-tailed distributions...

Kernel density estimation for heavy-tailed distributions... Kernel density estimation for heavy-tailed distributions using the Champernowne transformation Buch-Larsen, Nielsen, Guillen, Bolance, Kernel density estimation for heavy-tailed distributions using the

More information

On the Power of Tests for Regime Switching

On the Power of Tests for Regime Switching On the Power of Tests for Regime Switching joint work with Drew Carter and Ben Hansen Douglas G. Steigerwald UC Santa Barbara May 2015 D. Steigerwald (UCSB) Regime Switching May 2015 1 / 42 Motivating

More information

Statistical Estimation: Data & Non-data Information

Statistical Estimation: Data & Non-data Information Statistical Estimation: Data & Non-data Information Roger J-B Wets University of California, Davis & M.Casey @ Raytheon G.Pflug @ U. Vienna, X. Dong @ EpiRisk, G-M You @ EpiRisk. a little background Decision

More information

Lecture 17: Density Estimation Lecturer: Yihong Wu Scribe: Jiaqi Mu, Mar 31, 2016 [Ed. Apr 1]

Lecture 17: Density Estimation Lecturer: Yihong Wu Scribe: Jiaqi Mu, Mar 31, 2016 [Ed. Apr 1] ECE598: Information-theoretic methods in high-dimensional statistics Spring 06 Lecture 7: Density Estimation Lecturer: Yihong Wu Scribe: Jiaqi Mu, Mar 3, 06 [Ed. Apr ] In last lecture, we studied the minimax

More information

Distributed Estimation, Information Loss and Exponential Families. Qiang Liu Department of Computer Science Dartmouth College

Distributed Estimation, Information Loss and Exponential Families. Qiang Liu Department of Computer Science Dartmouth College Distributed Estimation, Information Loss and Exponential Families Qiang Liu Department of Computer Science Dartmouth College Statistical Learning / Estimation Learning generative models from data Topic

More information

WEIGHTED QUANTILE REGRESSION THEORY AND ITS APPLICATION. Abstract

WEIGHTED QUANTILE REGRESSION THEORY AND ITS APPLICATION. Abstract Journal of Data Science,17(1). P. 145-160,2019 DOI:10.6339/JDS.201901_17(1).0007 WEIGHTED QUANTILE REGRESSION THEORY AND ITS APPLICATION Wei Xiong *, Maozai Tian 2 1 School of Statistics, University of

More information

AUTOMATIC CONTROL COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET. Questions AUTOMATIC CONTROL COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET

AUTOMATIC CONTROL COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET. Questions AUTOMATIC CONTROL COMMUNICATION SYSTEMS LINKÖPINGS UNIVERSITET The Problem Identification of Linear and onlinear Dynamical Systems Theme : Curve Fitting Division of Automatic Control Linköping University Sweden Data from Gripen Questions How do the control surface

More information

STAT 6350 Analysis of Lifetime Data. Probability Plotting

STAT 6350 Analysis of Lifetime Data. Probability Plotting STAT 6350 Analysis of Lifetime Data Probability Plotting Purpose of Probability Plots Probability plots are an important tool for analyzing data and have been particular popular in the analysis of life

More information

Introduction to Regression

Introduction to Regression Introduction to Regression Chad M. Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/100 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression

More information

Penalized Exponential Series Estimation of Copula Densities

Penalized Exponential Series Estimation of Copula Densities Penalized Exponential Series Estimation of Copula Densities Ximing Wu Abstract The exponential series density estimator is advantageous to the copula density estimation as it is strictly positive, explicitly

More information

Linear Methods for Prediction

Linear Methods for Prediction Chapter 5 Linear Methods for Prediction 5.1 Introduction We now revisit the classification problem and focus on linear methods. Since our prediction Ĝ(x) will always take values in the discrete set G we

More information

Computational statistics

Computational statistics Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated

More information

BTRY 4090: Spring 2009 Theory of Statistics

BTRY 4090: Spring 2009 Theory of Statistics BTRY 4090: Spring 2009 Theory of Statistics Guozhang Wang September 25, 2010 1 Review of Probability We begin with a real example of using probability to solve computationally intensive (or infeasible)

More information

Modelling Non-linear and Non-stationary Time Series

Modelling Non-linear and Non-stationary Time Series Modelling Non-linear and Non-stationary Time Series Chapter 2: Non-parametric methods Henrik Madsen Advanced Time Series Analysis September 206 Henrik Madsen (02427 Adv. TS Analysis) Lecture Notes September

More information

Does k-th Moment Exist?

Does k-th Moment Exist? Does k-th Moment Exist? Hitomi, K. 1 and Y. Nishiyama 2 1 Kyoto Institute of Technology, Japan 2 Institute of Economic Research, Kyoto University, Japan Email: hitomi@kit.ac.jp Keywords: Existence of moments,

More information

A Novel Nonparametric Density Estimator

A Novel Nonparametric Density Estimator A Novel Nonparametric Density Estimator Z. I. Botev The University of Queensland Australia Abstract We present a novel nonparametric density estimator and a new data-driven bandwidth selection method with

More information

Calibration estimation using exponential tilting in sample surveys

Calibration estimation using exponential tilting in sample surveys Calibration estimation using exponential tilting in sample surveys Jae Kwang Kim February 23, 2010 Abstract We consider the problem of parameter estimation with auxiliary information, where the auxiliary

More information

Estimating and Testing Quantile-based Process Capability Indices for Processes with Skewed Distributions

Estimating and Testing Quantile-based Process Capability Indices for Processes with Skewed Distributions Journal of Data Science 8(2010), 253-268 Estimating and Testing Quantile-based Process Capability Indices for Processes with Skewed Distributions Cheng Peng University of Southern Maine Abstract: This

More information

Introduction to Rare Event Simulation

Introduction to Rare Event Simulation Introduction to Rare Event Simulation Brown University: Summer School on Rare Event Simulation Jose Blanchet Columbia University. Department of Statistics, Department of IEOR. Blanchet (Columbia) 1 / 31

More information

MINIMUM HELLINGER DISTANCE ESTIMATION IN A SEMIPARAMETRIC MIXTURE MODEL SIJIA XIANG. B.S., Zhejiang Normal University, China, 2010 A REPORT

MINIMUM HELLINGER DISTANCE ESTIMATION IN A SEMIPARAMETRIC MIXTURE MODEL SIJIA XIANG. B.S., Zhejiang Normal University, China, 2010 A REPORT MINIMUM HELLINGER DISTANCE ESTIMATION IN A SEMIPARAMETRIC MIXTURE MODEL by SIJIA XIANG B.S., Zhejiang Normal University, China, 2010 A REPORT submitted in partial fulfillment of the requirements for the

More information

Inverse Statistical Learning

Inverse Statistical Learning Inverse Statistical Learning Minimax theory, adaptation and algorithm avec (par ordre d apparition) C. Marteau, M. Chichignoud, C. Brunet and S. Souchet Dijon, le 15 janvier 2014 Inverse Statistical Learning

More information

GARCH Models Estimation and Inference

GARCH Models Estimation and Inference GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in

More information

Robust estimation of skew-normal distribution with location and scale parameters via log-regularly varying functions

Robust estimation of skew-normal distribution with location and scale parameters via log-regularly varying functions Robust estimation of skew-normal distribution with location and scale parameters via log-regularly varying functions Shintaro Hashimoto Department of Mathematics, Hiroshima University October 5, 207 Abstract

More information

Calibration Estimation of Semiparametric Copula Models with Data Missing at Random

Calibration Estimation of Semiparametric Copula Models with Data Missing at Random Calibration Estimation of Semiparametric Copula Models with Data Missing at Random Shigeyuki Hamori 1 Kaiji Motegi 1 Zheng Zhang 2 1 Kobe University 2 Renmin University of China Institute of Statistics

More information

11. Bootstrap Methods

11. Bootstrap Methods 11. Bootstrap Methods c A. Colin Cameron & Pravin K. Trivedi 2006 These transparencies were prepared in 20043. They can be used as an adjunct to Chapter 11 of our subsequent book Microeconometrics: Methods

More information

Empirical Likelihood

Empirical Likelihood Empirical Likelihood Patrick Breheny September 20 Patrick Breheny STA 621: Nonparametric Statistics 1/15 Introduction Empirical likelihood We will discuss one final approach to constructing confidence

More information

Stat 135, Fall 2006 A. Adhikari HOMEWORK 6 SOLUTIONS

Stat 135, Fall 2006 A. Adhikari HOMEWORK 6 SOLUTIONS Stat 135, Fall 2006 A. Adhikari HOMEWORK 6 SOLUTIONS 1a. Under the null hypothesis X has the binomial (100,.5) distribution with E(X) = 50 and SE(X) = 5. So P ( X 50 > 10) is (approximately) two tails

More information

A Conditional Approach to Modeling Multivariate Extremes

A Conditional Approach to Modeling Multivariate Extremes A Approach to ing Multivariate Extremes By Heffernan & Tawn Department of Statistics Purdue University s April 30, 2014 Outline s s Multivariate Extremes s A central aim of multivariate extremes is trying

More information

Adaptive Piecewise Polynomial Estimation via Trend Filtering

Adaptive Piecewise Polynomial Estimation via Trend Filtering Adaptive Piecewise Polynomial Estimation via Trend Filtering Liubo Li, ShanShan Tu The Ohio State University li.2201@osu.edu, tu.162@osu.edu October 1, 2015 Liubo Li, ShanShan Tu (OSU) Trend Filtering

More information

Introduction to Regression

Introduction to Regression Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression

More information

Interval Estimation for AR(1) and GARCH(1,1) Models

Interval Estimation for AR(1) and GARCH(1,1) Models for AR(1) and GARCH(1,1) Models School of Mathematics Georgia Institute of Technology 2010 This talk is based on the following papers: Ngai Hang Chan, Deyuan Li and (2010). Toward a Unified of Autoregressions.

More information

Entropy, Inference, and Channel Coding

Entropy, Inference, and Channel Coding Entropy, Inference, and Channel Coding Sean Meyn Department of Electrical and Computer Engineering University of Illinois and the Coordinated Science Laboratory NSF support: ECS 02-17836, ITR 00-85929

More information

FIXED-b ASYMPTOTICS FOR BLOCKWISE EMPIRICAL LIKELIHOOD

FIXED-b ASYMPTOTICS FOR BLOCKWISE EMPIRICAL LIKELIHOOD Statistica Sinica 24 (214), - doi:http://dx.doi.org/1.575/ss.212.321 FIXED-b ASYMPTOTICS FOR BLOCKWISE EMPIRICAL LIKELIHOOD Xianyang Zhang and Xiaofeng Shao University of Missouri-Columbia and University

More information

Nonparametric Methods

Nonparametric Methods Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis

More information

Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo

Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University

More information

High Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data

High Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data High Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data Song Xi CHEN Guanghua School of Management and Center for Statistical Science, Peking University Department

More information

ON THE CONSEQUENCES OF MISSPECIFING ASSUMPTIONS CONCERNING RESIDUALS DISTRIBUTION IN A REPEATED MEASURES AND NONLINEAR MIXED MODELLING CONTEXT

ON THE CONSEQUENCES OF MISSPECIFING ASSUMPTIONS CONCERNING RESIDUALS DISTRIBUTION IN A REPEATED MEASURES AND NONLINEAR MIXED MODELLING CONTEXT ON THE CONSEQUENCES OF MISSPECIFING ASSUMPTIONS CONCERNING RESIDUALS DISTRIBUTION IN A REPEATED MEASURES AND NONLINEAR MIXED MODELLING CONTEXT Rachid el Halimi and Jordi Ocaña Departament d Estadística

More information

Combining Interval and Probabilistic Uncertainty in Engineering Applications

Combining Interval and Probabilistic Uncertainty in Engineering Applications Combining Interval and Probabilistic Uncertainty in Engineering Applications Andrew Pownuk Computational Science Program University of Texas at El Paso El Paso, Texas 79968, USA ampownuk@utep.edu Page

More information

36. Multisample U-statistics and jointly distributed U-statistics Lehmann 6.1

36. Multisample U-statistics and jointly distributed U-statistics Lehmann 6.1 36. Multisample U-statistics jointly distributed U-statistics Lehmann 6.1 In this topic, we generalize the idea of U-statistics in two different directions. First, we consider single U-statistics for situations

More information

Asymptotic distribution of the sample average value-at-risk

Asymptotic distribution of the sample average value-at-risk Asymptotic distribution of the sample average value-at-risk Stoyan V. Stoyanov Svetlozar T. Rachev September 3, 7 Abstract In this paper, we prove a result for the asymptotic distribution of the sample

More information

Stat 451 Lecture Notes Numerical Integration

Stat 451 Lecture Notes Numerical Integration Stat 451 Lecture Notes 03 12 Numerical Integration Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapter 5 in Givens & Hoeting, and Chapters 4 & 18 of Lange 2 Updated: February 11, 2016 1 / 29

More information

Methods of Data Analysis Learning probability distributions

Methods of Data Analysis Learning probability distributions Methods of Data Analysis Learning probability distributions Week 5 1 Motivation One of the key problems in non-parametric data analysis is to infer good models of probability distributions, assuming we

More information

Nonparametric Estimation of Distributions in a Large-p, Small-n Setting

Nonparametric Estimation of Distributions in a Large-p, Small-n Setting Nonparametric Estimation of Distributions in a Large-p, Small-n Setting Jeffrey D. Hart Department of Statistics, Texas A&M University Current and Future Trends in Nonparametrics Columbia, South Carolina

More information

Ultra High Dimensional Variable Selection with Endogenous Variables

Ultra High Dimensional Variable Selection with Endogenous Variables 1 / 39 Ultra High Dimensional Variable Selection with Endogenous Variables Yuan Liao Princeton University Joint work with Jianqing Fan Job Market Talk January, 2012 2 / 39 Outline 1 Examples of Ultra High

More information

ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering

ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering ECE276A: Sensing & Estimation in Robotics Lecture 10: Gaussian Mixture and Particle Filtering Lecturer: Nikolay Atanasov: natanasov@ucsd.edu Teaching Assistants: Siwei Guo: s9guo@eng.ucsd.edu Anwesan Pal:

More information

Terminology for Statistical Data

Terminology for Statistical Data Terminology for Statistical Data variables - features - attributes observations - cases (consist of multiple values) In a standard data matrix, variables or features correspond to columns observations

More information

Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines

Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the

More information

Efficient estimation of a semiparametric dynamic copula model

Efficient estimation of a semiparametric dynamic copula model Efficient estimation of a semiparametric dynamic copula model Christian Hafner Olga Reznikova Institute of Statistics Université catholique de Louvain Louvain-la-Neuve, Blgium 30 January 2009 Young Researchers

More information

Nonparametric confidence intervals. for receiver operating characteristic curves

Nonparametric confidence intervals. for receiver operating characteristic curves Nonparametric confidence intervals for receiver operating characteristic curves Peter G. Hall 1, Rob J. Hyndman 2, and Yanan Fan 3 5 December 2003 Abstract: We study methods for constructing confidence

More information

O Combining cross-validation and plug-in methods - for kernel density bandwidth selection O

O Combining cross-validation and plug-in methods - for kernel density bandwidth selection O O Combining cross-validation and plug-in methods - for kernel density selection O Carlos Tenreiro CMUC and DMUC, University of Coimbra PhD Program UC UP February 18, 2011 1 Overview The nonparametric problem

More information

Incentives Work: Getting Teachers to Come to School. Esther Duflo, Rema Hanna, and Stephen Ryan. Web Appendix

Incentives Work: Getting Teachers to Come to School. Esther Duflo, Rema Hanna, and Stephen Ryan. Web Appendix Incentives Work: Getting Teachers to Come to School Esther Duflo, Rema Hanna, and Stephen Ryan Web Appendix Online Appendix: Estimation of model with AR(1) errors: Not for Publication To estimate a model

More information

Penalized Barycenters in the Wasserstein space

Penalized Barycenters in the Wasserstein space Penalized Barycenters in the Wasserstein space Elsa Cazelles, joint work with Jérémie Bigot & Nicolas Papadakis Université de Bordeaux & CNRS Journées IOP - Du 5 au 8 Juillet 2017 Bordeaux Elsa Cazelles

More information

Transformations and Bayesian Density Estimation

Transformations and Bayesian Density Estimation Transformations and Bayesian Density Estimation Andrew Bean 1, Steve MacEachern, Xinyi Xu The Ohio State University 10 th Conference on Bayesian Nonparametrics June 25, 2015 1 bean.243@osu.edu Transformations

More information

Density estimators for the convolution of discrete and continuous random variables

Density estimators for the convolution of discrete and continuous random variables Density estimators for the convolution of discrete and continuous random variables Ursula U Müller Texas A&M University Anton Schick Binghamton University Wolfgang Wefelmeyer Universität zu Köln Abstract

More information