A tour of kernel smoothing

Transcription

1 Tarn Duong Institut Pasteur October 2007

2 The journey up till now Bachelor, Univ. of Western Australia, Perth Researcher, Australian Bureau of Statistics, Canberra and Sydney PhD, Univ. of Western Australia, Perth 2005 Lecturer, Macquarie Univ., Sydney Post-doc, Univ. of New South Wales, Sydney 2007 present Post-doc, Institut Pasteur, Paris

3 Research interests Kernel smoothing Nonparametric statistics Statistical software

4 Today Kernel density estimation (KDE) 1st stage of inference (estimation) translation is Éstimation de densité à noyau Feature significance 2nd stage of inference (formal inference) translation is? extension of density estimation to significance testing

5

6 Kernel (1) NOT cell nucleus NOT kernel of an operating system NOT kernel/nullspace of a matrix A: {x : Ax = 0}

7 Kernel (2) Kernel K : R d R is K (x) 0 K (x) dx = 1 R d K is symmetric about 0

8 Kernel density estimation Let X 1, X 2,..., X n be a random sample drawn from a common density f. A kernel density estimate ˆf is where ˆf (x; H) = n 1 n K H (x X i ) i=1 K H (x X i ) = normal (Gaussian) pdf with mean X i, variance H H = bandwidth or window width (fenetre)

9 Graphical illustration Scaled kernels K H (x X i ) Kernel density estimate ˆf

10 Advantages of kernel density estimates non-parametric easy to construct easy to interpret suitable for multivariate data smooth, no discretisation effects no anchor points effects

11

12 Bandwidth selectors single most important factor effecting performance of ˆf ideal bandwidth selector: H 0 = argmin AMISE(H) H where AMISE = asymptotic R d E[ˆf (x; H) f (x)] 2 dx data-driven selector: Ĥ = argmin H AMISE(H)

13 Relative convergence rates (1) a data-driven selector Ĥ = argmin AMISE(H) converges to H H 0 with rate n α, α > 0 if vech(ĥ H 0) = O p (n α J) vech H 0 where O p is order in probability, J = matrix of ones, and [ ] a a b vech = b b c c

14 Relative convergence rates (2) Ĥ converges to H 0 with rate n α if MSE(Ĥ) = Var(Ĥ) + Bias(Ĥ) BiasT (Ĥ) = O(n 2α )(vech H 0 )(vech T H 0 )

15 Relative convergence rates (3) Easier(?!) to compute [ ]) (E Bias(Ĥ) = O vech H ( AMISE AMISE)(H 0 ) [ ]) (Var Var(Ĥ) = O vech H ( AMISE AMISE)(H 0 )

16 Table of convergence rates Convergence rate Selector d = 1 d > 1 Plug-in 1 (1994) n 4/13 n 4/(d+12) Plug-in 2 (2003) n 2/7 n 2/(d+6) CV 1 (1982, 1984) n 1/10 n min(d,4)/(2d+8) CV 2 (1994) n 1/10 n min(d,4)/(2d+8) CV 3 (1992, 2004) n 5/14 n 2/(d+6)

17

18 Software ks: R library available on CRAN comprehensive package for kernel density estimation and bandwidth selection

19 Flow cytometry (FACS) data (1) Data sample KDE x y x y

20 y Flow cytometry (FACS) data (2) Contour plot Wireframe plot Density function y x x

21 Independent citations in other fields Zago, A. and Dongili P. (2006) Bad loans and efficiency in Italian banks, Working paper no. 28, Università di Verona Fieberg, J. (2007) Kernel density estimators of home range: smoothing and the autocorrelation red herring. Ecology, 88, Peng T.G., Wang Y.H. and Wu T.H. (2007) Mean shift algorithm equipped with the intersection of confidence intervals rule for image segmentation. Pattern Recognition Letters, 28,

22

23 Features d = 1, 2: mode, valley, saddle-point, ridge etc. d > 2: mode

24 Modes and modal regions mode x of function f : R d R D f (x ) = 0, D 2 f (x ) < 0 D f (x ) = 0, eigenvalues λ 1 (x ), λ 2 (x ),..., λ d (x ) of D 2 f (x ) < 0 modal region M of f M = {x : D f (x) δ, ε λj (x) 0} δ, ε small positive

25 Kernel density derivative estimation density (zero-th derivative): ˆf (x; H) = n 1 gradient (first derivative): D f (x; H) = n 1 n K H (x X i ) i=1 n D K H (x X i ) i=1 curvature (second derivative): n D 2 f (x; H) = n 1 D 2 K H (x X i ) i=1

26 Kernel curvature estimators asymptotic distribution: vech D 2 f (x; H) approx. N(vech D 2 f (x), Σ(x)) local null hypothesis: H 0 (x) : vech D 2 f (x) = 0 null distribution: vech D 2 f (x; H) approx. N(0, Σ(x)) test statistic: W (x) = Σ(x) 1/2 vech D 2 f (x; H) 2 approx. χ 2 d(d+1)/2

27 Significant curvature regions extension of kernel density estimation suited to finding modal regions modal region estimate at significance level α: significant curvature region ˆM = {x : W (x) χ 2 d(d+1)/2;1 α } α is adjusted significance level to account for multiple hypothesis tests

28 Software feature: R library available on CRAN

29 y Flow cytometry (FACS) data (3) Density estimate Modal regions estimates y x x

30

31 Summary Multivariate kernel density estimators theoretical development of optimal bandwidth selectors software implementation Feature significance some theoretical development of multivariate modal region estimation software implementation

32 Future directions Comparing two kernel density estimators Optimal bandwidth selection for kernel density derivative estimators

33 Acknowledgements Kernel density estimation Prof. Martin Hazelton, then Univ. of Western Australia, now at Massey Univ. (New Zealand), as PhD supervisor Feature significance Dr Inge Koch, Univ. of New South Wales (Australia), Prof. Matt Wand, then Univ. of New South Wales (Australia), now at Univ. of Wollongong (Australia)