Nonparametric Function Estimation with Infinite-Order Kernels

Transcription

1 Nonparametric Function Estimation with Infinite-Order Kernels Arthur Berg Department of Statistics, University of Florida March 15, 2008

2 Kernel Density Estimation (IID Case) Let X 1,..., X n iid density f. R K(x) dx = 1 R K2 (x) dx < Definition (Kernel Order) ˆf (x) = 1 nh The order of kernel K is the largest integer p such that x p K(x) dx < and R x i K(x) dx = 0 n ( ) x Xj K h j=1 (i = 1,... p 1). Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 2/ 28

3 Infinite-Order Kernels We shall construct a class of kernels K(x) that satisfies x j K(x) = 0 for all j = 1, 2, 3,.... Hence K(x) will be of infinite order. Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 3/ 28

4 Constructing Flat-Top Lag-Windows and Kernels Definition (Flat-top Lag-Window) A function λ : R k R that satisfies (i) λ(x) 1 for all x b; (ii) λ(x) 1 for all x; (iii) λ has compact support. Then λ is called a flat-top lag-window. Definition ( Flat-top Kernel) A flat-top kernel K is the Fourier transformation of λ; K(x) = λ(ω)e iω x dω R k Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 4/ 28

5 Flat-top Examples Figure: Two Dimensional Lag-Windows Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 5/ 28 Figure: One Dimensional Flat-top Lag-Windows Figure: One Dimensional Flat-top Kernels

6 Infinite-Order Kernels Work! N RMSE flat-top RMSE Gaussian Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 6/ 28

7 KDE Polyspectra Bandwidth Selection Censored Deficiency Simulations Symmetry Literature Review Estimation with Flat-top Kernels univariate iid (Devroye 1992) univariate iid (Politis and Romano 1993) nonparametric regression (Mcmurry and Politis 2004) spectral density (Politis and Romano 1995) spectral density matrices (Politis 2005) multivariate iid (Politis and Romano 1999) homogeneous random fields (Politis and Romano 1996) bandwidth algorithm for flat-tops (Politis 2001) Arthur Berg sth -order spectra plus bandwidth (Berg and Politis 2007) censored data (Berg and Politis 2007) cumulative distribution function (Berg and Politis 2008) Nonparametric Function Estimation with Infinite-Order Kernels 7/ 28

8 Stationary Time Series Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 8/ 28

9 Spectrum and Bispectrum X 1,..., X n second-order stationary time series means Third-order stationary means C(j) = cov(x t, X t+j ) is free of t. C(τ 1, τ 2 ) = E [(X t µ)(x t+τ1 µ)(x t+τ2 µ)] is free of t. Fourth and higher-order stationary means the joint cumulant is free of t. The spectral density, or spectrum, is the Fourier transform of C(j), i.e. f (ω) = 1 C(j)e ijω. 2π j= Likewise, the bispectral density, or bispectrum, is f (ω 1, ω 2 ) = 1 (2π) 2 C(τ 1, τ 2 )e iτ 1ω 1 iτ 2 ω 2. τ 1 = τ 2 = Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 9/ 28

10 Spectral Density Estimation f (ω) = 1 2π j= C(j)e ijω f (ω) = 1 2π j= λ (h j) Ĉ(j)e ijω f (ω 1, ω 2 ) = 1 (2π) 2 f (ω 1, ω 2 ) = 1 (2π) 2 Definition (Lag-window Order) τ 1 = τ 2 = τ 1 = τ 2 = C(τ 1, τ 2 )e iτ 1ω 1 iτ 2 ω 2. λ (h τ 1, h τ 2 ) Ĉ(τ 1, τ 2 )e iτ 1ω 1 iτ 2 ω 2 λ has order p if λ(τ ) = 1 + a τ p + O( τ p ) as τ 0. Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 10/ 28

11 Bias and Variance of Kernel Estimates Suppose λ has order p = 2 and f is sufficiently smooth, then ) bias (ˆf = O(h 2 ) and ) ( ) 1 var (ˆf = O. nh Thus the optimal bandwidth is h n 1/5 in which case ) ) MSE (ˆf = O (n 4 5 Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 11/ 28

12 Bias and Variance of Kernel Estimates Suppose λ has order p = and f is sufficiently smooth, then ) bias (ˆf = O(e a/h ) and ) ( ) 1 var (ˆf = O. nh Thus the optimal bandwidth is h A/ log n in which case ) MSE (ˆf = log n n Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 11/ 28

13 General Theorem with Flat-Top Estimators Theorem (Berg, Politis 2007) Let ˆf be a flat-top estimator. (i) Assume τ Z k τ r C(τ ) <. Letting h An 1/(2r+k) gives } ) ) sup bias {ˆf (x) = o (n r 2r+k and MSE(ˆf (x)) = O (n 2r 2r+k x (ii) Assume C(τ ) De d τ. Letting h A/ log n gives } ( ) 1 ) ( ) log n sup bias {ˆf (x) = O n and MSE (ˆf (x) = O x n (iii) Assume C(τ ) = 0 when τ q. Letting h b/q gives } ( 1 ) ( ) 1 sup bias {ˆf (x) = O and MSE (ˆf (x) = O x n) n Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 12/ 28

14 General Theorem with Flat-Top Estimators Theorem (Berg, Politis 2007) Let ˆf be a flat-top estimator. (i) Assume τ Z k τ r C(τ ) <. Letting h An 1/(2r+k) gives } ) ) sup bias {ˆf (x) = o (n r 2r+k and MSE(ˆf (x)) = O (n 2r 2r+k x (ii) Assume C(τ ) De d τ. Letting h A/ log n gives } ( ) 1 ) ( ) log n sup bias {ˆf (x) = O n and MSE (ˆf (x) = O x n (iii) Assume C(τ ) = 0 when τ q. Letting h b/q gives } ( 1 ) ( ) 1 sup bias {ˆf (x) = O and MSE (ˆf (x) = O x n) n Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 12/ 28

15 Normalized ACF One variable k variables ρ(τ ) = C(τ) C(0) ˆρ(τ) = Ĉ(τ) Ĉ(0) ρ(τ ) = C(τ ) C(0) k+1 2 ˆρ(τ ) = Ĉ(τ ) Ĉ(0) k+1 2 Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 13/ 28

16 Specialized Bandwidth Selection Procedure BANDWIDTH SELECTION ALGORITHM FOR FLAT-TOP ESTIMATORS (in picture) Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 14/ 28

17 Specialized Bandwidth Selection Procedure BANDWIDTH SELECTION ALGORITHM FOR FLAT-TOP ESTIMATORS (in picture) Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 14/ 28

18 General Theorem for the Bandwidth Selection Algorithm Theorem (Berg, Politis 2007) (i) Assume C(τ ) A τ d. Then ( ) 1 ĥ P log n 2d à ; n here A P B means A/B 1 in probability. (ii) Assume C(τ ) A ξ τ. Then ĥ P Ã/ log N. (iii) Assume C(τ ) = 0 when τ q. Then ĥ P b/q. Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 15/ 28

19 Density Estimation of Censored Data Lifetime variables X 1,..., X n iid density f Censoring variables Y 1,..., Y n iid density g Observe Z i = min(x i, Y i ) and δ i = 1 [Xi Y i ] {0, 1} (i = 1,..., n). Convolve kernel K with the Kaplan-Meier estimator: ˆf (x) = 1 h n ( ) x Z(j) s j K h s j is the height of the jump of the Kaplan-Meier estimator at Z (j). j=1 Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 16/ 28

20 Transitioning to the Censored Case K(x) = 1 2π 1 h K(x/h) = 1 2π ˆf (x) = 1 h n ( ) x Xj s j K h j=1 κ(t)e itx dt. κ(th)e itx dt = 1 n h s j κ(th)e it(x Xj) dt h 2π j=1 = 1 n s j e itx j κ(th)e itx dt 2π = 1 2π j=1 ˆφ(t)κ(th)e itx dt Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 17/ 28

21 Smoothness Conditions on f Let φ(s) be the characteristic function corresponding to f, i.e. φ(s) = R k e iω s f (ω) dω. Censored : Kernel K Dependent : Lag-Window λ A(r): s r φ(s) < τ Z k τ r C(τ ) <. B: φ(s) De d s C(τ ) De d τ C: φ(s) = 0 when s q C(τ ) = 0 when τ q Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 18/ 28

22 Smoothed Kaplan-Meier Estimator Estimation of the CDF: ˆF h (t) = t ˆf h (x) dx = 1 n n j=1 where K(t) = t K(x) dx. Estimation of the Survival Function: Ŝ h (t) = ( ) t Xj s j K h ( ) t Xj K h where s j is the height of the jump of the Kaplan-Meier estimator at X j. var [ˆF h (t) ] = F(t)[1 F(t)] n ( 2f (t) ) ( ) h h u K(u)K(u) du n + o. n Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 19/ 28

23 Theorem (Berg, Politis 2007) Let Ŝ h (t) be a flat-top estimator of the survival function. (i) Suppose assumption A(r) holds, then ( bias (Ŝh (t)) = o h r+1 ) = o sup t R ) (n r+1 2r+1 when h an β where a is any positive constant and β = (2r + 1) 1. (ii) Suppose assumption B holds, then sup t R ) ( bias (Ŝh (t) = O he d/h) = o when h a/ log n where a < 2d is constant. (iii) Suppose assumption C holds and h 1/b, then bias (Ŝh (t)) = 0. sup t R ( e d/h) = o ( ) 1. n log n Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 20/ 28

24 Censored Simulations N RMSE flat-top RMSE muhaz RMSE logspline Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 21/ 28

25 CDF and Survival Simulations Table: Comparisons with iid standard normal data and censored Weibull data t = 1.5 t = 0 t = 1.5 n MSE * infinite MSE * second MSE * EDF t =.75 t = 1.25 t = 1.75 n MSE * infinite MSE * second MSE * Kaplan-Meier * MSE values are blown up by 10 3 for easier comparison. Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 22/ 28

26 Symmetries of the Bivariate ACF and Bispectrum Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 23/ 28

27 Symmetries of ACF One variable: C(x) = C( x) Two variables: Eq. 2 + Eq. 3 Eq. 6 C(x, y) 1 = C(x, y) 2 = C( x, y x) 3 = C(y, x) 4 = C(x y, y) 5 = C( y, x y) 6 = C(y x, x) C(x, y) = C(y, x) = C( x, y x) suffices Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 24/ 28

28 Constructing the Symmetries from Permutations Each equation corresponds to a permutation. For simplicity, assume E[X t ] = 0. The equation corresponding to the permutation σ = (12) is C(x, y) = E[X t X t+x X t+y ] (12) = E[X t+x X t X t+y ] = E[X t X t x X t x+y ] = C( x, y x) Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 25/ 28

29 A Group Representation Define ψ : R 3 R 2 given by ψ(a, b, c) (b a, c a). Take σ = (12), then ( ) σ ψ 1 0 (x, y) (0, x, y) (x, 0, y) ( x, y x) 1 1 ( ) ( ) e C(x, y) (13) C(x y, y) ( ) ( ) (12) C( x, y x) (123) C( y, x y) ( ) ( ) (23) C(y, x) (132) C(y x, x) Suppose σ = (12) and τ = (13), then γ = (132) = στ and ( ) ( ) ( ) ρ(γ) = = = ρ(σ)ρ(τ) Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 26/ 28

30 Implications of the Representation Theorem (Berg 2007) The mapping ρ : S n GL n 1 (R), described above, is a faithful group representation. 1 Symmetrizing lag-windows generalizing current constructions Current construction: f (x, y) = f (x)f (y)f (x y) f (x, y) = h(f (x, y), f ( x, y x), f (y, x), f (x y, y), f ( y, x y), f (y x, x)) where h is any symmetric function of its six arguments. Figure: f with h = x i, h = max(x i ), h = min(x i ), and f (x, y) = (1 x 2 y 2 ) +. Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 27/ 28

31 Implications of the Representation 2 Generalization of the Gabr-Rao optimal window Theorem (Berg 2007) k 1 Λ opt (ω) = α 1 β i=1 ω 2 i + i<j ω i ω j Let Λ(ω) be any nonnegative kernel that integrates to one and satisfies all the necessary symmetries. Also assume ωj 2 Λ(ω) dω = ωj 2 Λ opt (ω) dω R k 1 R k 1 for j = 1,..., n 1. Then Λ L2 Λ opt L2. + Arthur Berg Nonparametric Function Estimation with Infinite-Order Kernels 28/ 28