Saddlepoint-Based Bootstrap Inference in Dependent Data Settings

Transcription

1 Saddlepoint-Based Bootstrap Inference in Dependent Data Settings Alex Trindade Dept. of Mathematics & Statistics, Texas Tech University Rob Paige, Missouri University of Science and Technology Indika Wickramasinghe, Eastern New Mexico University (former student) Pratheepa Jeganathan, Texas Tech University (current student) June 2014 SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 1 Missour / 21

2 Outline 1 Overview of SPBB inference: Saddlepoint-Based Bootstrap An approximate parametric bootstrap for scalar parameter θ 2 Application 1: Spatial Regression Models Classic application of SPBB Better performance than asymptotic-based CIs 3 Application 2: MA(1) Model Challenging; extend methodology in two directions Extension 1: Non-Monotone QEEs Extension 2: Non-Gaussian QEEs SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 2 Missour / 21

3 Saddlepoint-Based Bootstrap (SPBB) Inference Pioneered by Paige, Trindade, & Fernando (SJS, 2009): SPBB: an approximate percentile parametric bootstrap; replace (slow) MC simulation with (fast) saddlepoint approx (SPA); estimators are roots of QEE (quadratic estimating equation); enjoys near exact performance; orders of magnitude faster than bootstrap; may be only alternative to bootstrap if no exact or asymptotic procedures; Idea: relate distribution of root of QEE Ψ(θ) to that of estimator ˆθ; under normality on data have closed form for MGF of QEE; use to saddlepoint approximate distribution of estimator (PDF or CDF); can pivot CDF to get a CI... numerically! leads to 2nd order accurate CIs, coverage error is O(n 1 ). SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 3 Missour / 21

4 SPBB: An Approximate Parametric Bootstrap Intractable! (And bootstrap too expensive...) F ˆθ ( ˆθ obs ) (θ L, θ U ) ˆθ solves pivot Ψ(θ) = 0 ˆF Ψ( ˆθ obs ) (0) Ψ(θ) monotone F ˆθ ( ˆθ obs ) = F Ψ( ˆθ obs ) (0) SPA via MGF of Ψ(θ) SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 4 Missour / 21

5 Application 1: Spatial Regression Models (Lattice Data) Jeganathan, Paige, & Trindade (under review) Spatial process y = [y(s 1 ),..., y(s n )] observed at sites {s 1,..., s n }. Under stationarity & isotropy, correlation modeled via spatial dependence parameter ρ and spatial weights matrix W. 3 main correlation structures for the regression model y = X β + z, z N(0, σ 2 V ρ ) SAR: z = ρw z + ε, with V ρ = (I n ρw ) 1 (I n ρw ) 1 CAR: E[z(s i ) z(s j ) : s j N(s i )] = ρ w ij z(s j ), with V ρ = (I n ρw ) 1 SMA: z = ρw ε + ε, with V ρ = (I n + ρw )(I n + ρw ). SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 5 Missour / 21

6 QEEs for ML and REML Estimators of ρ IRWGLS estimate for z (fixed ρ): ẑ r = P C (X ) y, with y N(X β, σ 2 V ρ ) = r N(0, σ 2 V ρ P T C (X ) ) Leads to following QEEs for estimators ˆρ ML & ˆρ REML : [ ( ) Ψ ML (ρ) = r Tr Vρ 1 V ρ Vρ 1 nvρ 1 ] V ρ Vρ 1 r [ ( ) ( Ψ REML (ρ) = r T Tr Vρ 1 V ρ Vρ 1 Tr P C (X ) V ρ V 1 ρ (n q) V 1 ρ Theorem: ML QEE is biased; REML QEE is unbiased. ) V 1 ρ V ρ V 1 ρ ] r SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 6 Missour / 21

7 Approximations for Distribution of ˆρ ML : CAR Model ρ = 0.05 and n = 36 ρ = 0.05 and n = 100 Density ρ = 0.15 and n = 36 Density Density ρ = 0.15 and n = 100 Density ρ = 0.2 and n = 36 ρ = 0.2 and n = 100 Density Density Asymptotic Normal (solid) Saddlepoint Approx (dashed) Empirical (histogram) SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 7 Missour / 21

8 Empirical Coverage Probabilities and Average Lengths of 95% SPBB & Asymptotic CIs for ˆρ ML : CAR Model Coverages Lengths Sample size ρ 0 =0.05 ρ 0 =0.15 ρ 0 =0.2 SPBB ASYM SPBB ASYM SPBB ASYM n= n= n= Sample size ρ 0 =0.05 ρ 0 =0.15 ρ 0 =0.2 SPBB ASYM SPBB ASYM SPBB ASYM n= n= n= SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 8 Missour / 21

9 Real Dataset 1: Mercer & Hall (1911) Wheat Yield Yield of grain collected in summer of 1910 over n = 500 plots (20 25 grid). Mean trend removed via median polish. Available in R package spdep as wheat. Cressie (1985, 1993): use spatial binary weights matrix W with polynomial neighborhood structure in SAR model. Table: MLEs and 95% SPBB & ASYM CIs for ρ 0. Estimation Method SAR model CAR model SMA model MLE SPBB 95% CI (0.477, 0.726) (0.067, 0.084) (0.055, 0.098) ASYM 95% CI (0.478, 0.727) (0.069, 0.088) (0.055, 0.098) SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June Rob 2014Paige, 9 Missour / 21

10 Real Dataset 2: Eire county blood group A percentages Available in R package spdep as eire. Percentage of a sample with blood type A collected over n = 26 counties in Eire. Covariates: towns (towns/unit area) and pale (1=within, 0=beyond). Used by Cliff & Ord (1973) to illustrate spatial dependence in SAR and CAR models (binary weights matrix W with neighborhood structure as in spdep). Estimation Method SAR model CAR model SMA model MLE SPBB 95% CI (-0.190, 0.769) (-0.167, 0.195) (-0.078, 0.209) ASYM 95% CI ( 0.151, 0.776) ( 0.128, 0.283) ( 0.083, 0.192) SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 10 Missour / 21

11 Application 2: MA(1) Model Paige, Trindade, & Wickramasinghe (AISM, 2014) Challenging...; extend methodology in two directions. Extension 1: Non-Monotone QEEs Problem: non-monotone QEEs invalidate SPBB Solution: double-spa & importance sampling Extension 2: Non-Gaussian QEEs Problem: key to SPBB is QEE with tractable MGF Solution: elliptically contoured distributions, and some tricks SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 11 Missour / 21

12 The MA(1): World s Simplest Model? Model: X t = θ 0 Z t 1 + Z t, Z t iid (0, σ 2 ), θ 0 1 Uses: special case of more general ARMA models; perhaps most useful in testing if data has been over-differenced... if we difference WN we get MA(1) with θ 0 = 1 X t = Z t = Y t X t X t 1 = Z t Z t 1 connection with unit-root tests in econometrics (Tanaka, 1990, Davis et al., 1995, Davis & Dunsmuir, 1996, Davis & Song, 2011). Inference: complicated... common estimators (MOME, LSE, MLE) have mixed distributions, point masses at ±1 and continuous over ( 1, 1); LSE & MLE are roots of polynomials of degree 2n. SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 12 Missour / 21

13 Unification of Parameter Estimators Theorem For θ < 1, MOME, LSE, and MLE are all roots of QEE, Ψ(θ) = x A θ x, where symmetric matrix A θ in each case is MOME: (QEE is monotone) LSE: (QEE not monotone...) MLE: (QEE not monotone...) A θ = (1 + θ 2 )J n 2θI n A θ = Ω 1 θ [J n + 2θI n ]Ω 1 θ A θ = function(θ, I n, J n, Ω 1 θ ) SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 13 Missour / 21

14 SPA densities of estimators: MOME, LSE, MLE, AN n=10, θ=0.4 n=10, θ= AN MLE CLSE MOME n=20, θ=0.4 n=20, θ= SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 14 Missour / 21

15 95% CI Coverages & Lengths for MOME (Gaussian Noise) Settings Coverage Probability Average Length n θ 0 SPBB Boot AN SPBB Boot AN SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 15 Missour / 21

16 Extension 1: Non-Monotone Estimating Equations Monotonicity of QEE is key (Daniels, 1983). Skovgaard (1990) & Spady (1991) give expression for PDF of ˆθ where Jacobian does not require monotonicity of Ψ(t) in t; but involves an intractable conditional expectation... Solution: double-spa & importance sampling. SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 16 Missour / 21

17 Example: Density of MLE (Gaussian Noise) n=10, θ=0.4 n=10, θ= Skovgaard (solid) Daniels (dashed) Empirical (histogram) n=20, θ=0.4 n=20, θ= SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 17 Missour / 21

18 Extension 2: Non-Gaussian QEEs General Problem with SPBB: need QEEs with tractable MGF... One solution: elliptically contoured (EC) distributions. Relies on appropriate weighting function w(t) (Provost & Cheong, 2002). Theorem With M N the MVN MGF: M EC (s; µ, Σ) = 0 w (t) M N (s; µ, Σ/t) dt SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 18 Missour / 21

19 SPBB approx PDFs of MOME in Laplace MA(1) n=5, θ=0.4 n=10, θ=0.4 Density Density n=5, θ=0.8 n=10, θ=0.8 Density Density SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 19 Missour / 21

20 Key References Butler, R.W. (2007), Saddlepoint Approximations With Applications, New York: Cambridge University Press. Paige, R.L. and Trindade, A.A. (2008), Practical Small Sample Inference for Single Lag Subset Autoregressive Models, J. Statist. Plan. Inf., 138, Paige, R.L., Trindade, A.A. and Fernando, P.H. (2009), Saddlepoint-based bootstrap inference for quadratic estimating equations, Scand. J. Stat., 36, Paige, R.L., Trindade, A.A., and Wickramasinghe, R.I.P., Extensions of Saddlepoint-Based Bootstrap Inference, Ann. Instit. Statist. Math., to appear. Jeganathan, P., Paige, R.L. and Trindade, A.A., Saddlepoint-Based Bootstrap Inference for Spatial Dependence in the Lattice Process, under review. SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 20 Missour / 21

21 SPBB Details: Key Steps Estimator ˆθ of θ 0 solves QEE Ψ(θ) = x A θ x = 0 Assume: x N(µ, Σ) = closed-form for MGF of QEE. QEE monotone (e.g., decreasing) in θ implies: F ˆθ (t) = P( ˆθ t) = P (Ψ(t) 0) = F Ψ(t) (0) Nuisance parameter λ: substitute conditional MLE, ˆλ θ. Now: accurately approximate distribution of ˆθ via SPA F ˆθ (t; θ 0, λ 0 ) ˆF ˆθ (t; θ 0, ˆλ ) θ0 = ˆF Ψ(t) (0; θ 0, ˆλ ) θ0 CI (θ L, θ U ) produced by pivoting SPA of CDF ˆF Ψ( ˆθ obs ) (0; θ L, ˆλ θl ) = 1 α 2, ˆF Ψ( ˆθ obs ) (0; θ U, ˆλ θu ) = α 2 SPBB ( Inference Dept. ofunder Mathematics Dependence & Statistics, Texas Tech University June 2014 Rob Paige, 21 Missour / 21