Nonparametric Inference In Functional Data

Transcription

1 Nonparametric Inference In Functional Data Zuofeng Shang Purdue University Joint work with Guang Cheng from Purdue Univ.

2 An Example Consider the functional linear model: Y = α + where 1 0 X(t)β(t)dt + ɛ, β W2 m (0, 1), the Sobolev space of order m X is a random process ɛ is zero-mean error

3 General Aim In this talk, we address the following questions in a unified framework: how to construct confidence interval for the regression mean µ = α x(t)β(t)dt? how to construct prediction interval for Y future? how to test H 0 : β = β 0 versus H 1 : β β 0?

4 General Aim In this talk, we address the following questions in a unified framework: how to construct confidence interval for the regression mean µ = α x(t)β(t)dt? how to construct prediction interval for Y future? how to test H 0 : β = β 0 versus H 1 : β β 0?

5 General Aim In this talk, we address the following questions in a unified framework: how to construct confidence interval for the regression mean µ = α x(t)β(t)dt? how to construct prediction interval for Y future? how to test H 0 : β = β 0 versus H 1 : β β 0?

6 General Aim In this talk, we address the following questions in a unified framework: how to construct confidence interval for the regression mean µ = α x(t)β(t)dt? how to construct prediction interval for Y future? how to test H 0 : β = β 0 versus H 1 : β β 0?

7 Literature Review The existing methods for inference rely on functional principle component analysis (FPCA), which requires the covariance kernel and reproducing kernel to share common ordered eigenfunctions, i.e., perfectly aligned; Müller and Stadtmüller (2005), Cai and Hall (2006), Hall and Horowitz (2007), etc. There is a lack of unified treatment for various inference problems such as confidence/prediction interval construction, (adaptive) hypothesis testing, and functional contrast testing.

8 Literature Review The existing methods for inference rely on functional principle component analysis (FPCA), which requires the covariance kernel and reproducing kernel to share common ordered eigenfunctions, i.e., perfectly aligned; Müller and Stadtmüller (2005), Cai and Hall (2006), Hall and Horowitz (2007), etc. There is a lack of unified treatment for various inference problems such as confidence/prediction interval construction, (adaptive) hypothesis testing, and functional contrast testing.

9 Model and Assumptions: Model: Y i = α X i (t)β(t)dt + ɛ i, where (Y 1, X 1 ),..., (Y n, X n ) are iid samples and E{ɛ i } = 0, E{ɛ 2 i } = 1 Functional parameter: β W2 m (0, 1), the m-order Sobolev space Covariance function: C(s, t) = E{X(s)X(t)} satisfies 1 0 C(s, t)β(s)ds = 0 β = 0

10 Model and Assumptions: Model: Y i = α X i (t)β(t)dt + ɛ i, where (Y 1, X 1 ),..., (Y n, X n ) are iid samples and E{ɛ i } = 0, E{ɛ 2 i } = 1 Functional parameter: β W2 m (0, 1), the m-order Sobolev space Covariance function: C(s, t) = E{X(s)X(t)} satisfies 1 0 C(s, t)β(s)ds = 0 β = 0

11 Model and Assumptions: Model: Y i = α X i (t)β(t)dt + ɛ i, where (Y 1, X 1 ),..., (Y n, X n ) are iid samples and E{ɛ i } = 0, E{ɛ 2 i } = 1 Functional parameter: β W2 m (0, 1), the m-order Sobolev space Covariance function: C(s, t) = E{X(s)X(t)} satisfies 1 0 C(s, t)β(s)ds = 0 β = 0

12 FPCA Estimation Sample covariance function: Ĉ(s, t) = 1 n n (X i (s) X(s))(X i (t) X(t)) i=1 KarhunenLoéve decomposition: C(s, t) = k=1 λ kψ k (s)ψ k (t) with λ 1 λ 2... Ĉ(s, t) = k=1 λ k ψk (s) ψ k (t) with λ 1 λ 2... Estimate β by β = b 1 ψ1 + b 2 ψ2 + + b kn ψkn, where b j are estimated basis coefficients.

13 FPCA Estimation Sample covariance function: Ĉ(s, t) = 1 n n (X i (s) X(s))(X i (t) X(t)) i=1 KarhunenLoéve decomposition: C(s, t) = k=1 λ kψ k (s)ψ k (t) with λ 1 λ 2... Ĉ(s, t) = k=1 λ k ψk (s) ψ k (t) with λ 1 λ 2... Estimate β by β = b 1 ψ1 + b 2 ψ2 + + b kn ψkn, where b j are estimated basis coefficients.

14 FPCA Estimation Sample covariance function: Ĉ(s, t) = 1 n n (X i (s) X(s))(X i (t) X(t)) i=1 KarhunenLoéve decomposition: C(s, t) = k=1 λ kψ k (s)ψ k (t) with λ 1 λ 2... Ĉ(s, t) = k=1 λ k ψk (s) ψ k (t) with λ 1 λ 2... Estimate β by β = b 1 ψ1 + b 2 ψ2 + + b kn ψkn, where b j are estimated basis coefficients.

15 Penalized Estimation: where ( α, β) = arg l n,λ (α, β) = 1 2n + λ 2 min α R,β W m 2 n (Y i α i=1 1 0 (0,1) l n,λ(α, β), 1 0 β (m) (t) 2 dt. X i (t)β(t)dt) 2

16 Advantage of Penalized Estimation No perfect alignment assumption Provides a unified framework for inference Easy to make nonparametric inference within regularization framework Estimation performance is better

17 Advantage of Penalized Estimation No perfect alignment assumption Provides a unified framework for inference Easy to make nonparametric inference within regularization framework Estimation performance is better

18 Advantage of Penalized Estimation No perfect alignment assumption Provides a unified framework for inference Easy to make nonparametric inference within regularization framework Estimation performance is better

19 Advantage of Penalized Estimation No perfect alignment assumption Provides a unified framework for inference Easy to make nonparametric inference within regularization framework Estimation performance is better

20 A Graphical Comparison of FPCA and Penalized Estimation: Cai and Yuan (2012) k 0 controls the alignment between covariance and reproducing kernels. Larger value of k 0 yields more misalignment. Roughness Regularization Functional PCA Excess Risk k0= 5 k0= 10 k0= 15 k0= 20 Excess Risk Relative Efficiency n n

21 Assumption: Simultaneous Diagonalization There exists functions ϕ ν and nondecreasing sequences ρ ν ν 2k for some k > 0 such that for any ν, µ 1, C(s, t)ϕ ν (s)ϕ µ (t)dsdt = δ νµ, and 1 0 ϕ (m) ν (t)ϕ (m) µ (t)dt = ρ ν δ νµ. Furthermore, any β W m 2 (0, 1) satisfies β = ν b νϕ ν for some real sequence b ν.

22 Construction of CI Let µ 0 = α x 0(t)β(t)dt be the regression mean at X = x 0. The 95% confidence interval for µ 0 is CI : µ 0 ± 1.96σ n / n, where µ 0 = α x 0(t) β(t)dt, σ 2 n = 1 + ν x ν = 1 0 x 0(t)ϕ ν (t)dt. x 2 ν 1+λρ ν,

23 Construction of PI Let Y 0 be future response generated from Y 0 = µ 0 + ɛ, then the 95% prediction interval for Y 0 is P I : µ 0 ± σ 2 n/n.

24 Theoretical Validity Theorem If ɛ is sub-exponential, the true function β 0 is suitably smooth, and λ is properly tuned, e.g., λ n k/(2k+1). Then as n, P (µ 0 CI) 0.95, and P (Y 0 P I) 0.95.

25 Penalized Likelihood Ratio Test Testing hypotheses H 0 : α = α 0, β = β 0 versus H 1 : H 0 is not true. Define the penalized likelihood ratio test (PLRT) P LRT n = l n,λ (α 0, β 0 ) l n,λ ( α, β), where ( α, β) is the penalized MLE.

26 Wilks Phenomenon Wilks phenomenon means that the null limit distribution of the likelihood ratio is free of any nuisance parameters and design distribution. Theorem Suppose H 0 holds and E{ɛ 4 } <, and λ is suitably tuned, e.g., λ n 4k/(4k+1). Then where σ 2 = 2nσ 2 P LRT n d χ 2 un, 0 (1 + x2k ) 1 dx 0 (1 + x2k ) 2 dx, u n = 1 cλ 1 2k c is constant free of α 0, β 0, distribution of X. ( 1 0 (1 + x2k ) 1 dx) 2 ( 1 0 (1 + x2k ) 2 dx),

27 Minimax Property of PLRT Suppose we want to test H 0 : β = 0, but the following local alternative hypothesis is true: H 1n : β = β n, where β n satisfies β n L 2 cn 2k/(4k+1). Theorem For arbitrary ε > 0, there exist c such that for any n 1: inf P βn (reject H 0 ) 1 ε. β n W2 m(0,1): βn L 2 cn 2k/(4k+1)

28 An Example: Standard Brownian Motion When m = 2 (cubic spline) and X is Brownian motion with covariance function C(s, t) = min{s, t}, s, t (0, 1), we have σ and u n 0.31λ 1/6. Therefore, 2n(1.08) P LRT n d χ 2 un.

29 An Adaptive Testing Procedure Based on Likelihood Ratio If the smoothness degrees of both X and β are unknown, how well can we do? We will propose a testing procedure adaptive to these smoothness degrees and show that our procedure achieves the minimax rate of testing.

30 Let P LRT (k) be the penalized likelihood ratio test associated with k, and τ k = P LRT (k) E{P LRT (k)} V ar(p LRT (k)), k = 1, 2,..., k n. Define AT = B n ( max 1 k k n τ k B n ), where B n satisfies 2πB 2 n exp(b 2 n) = k 2 n.

31 Size of the Test A valid test should achieve the correct size. Theorem Under H 0 : β = 0, if k n (log n) d 0, for some constant d 0 (0, 1/2), then for any γ (0, 1), P (AT c γ ) 1 γ, as n, where c γ = log( log(1 γ)).

32 Adaptive Minimax Rate Suppose k is the true value of k. Let δ(n, k ) = n 2k /(4k +1) (log log n) k /(4k +1). Theorem Suppose k n (log n) d 0, for some constant d 0 (0, 1/2). Then, for any ε (0, 1), there exists c > 0 s.t. for any n 1, inf P β(reject H 0 ) 1 ε. β L 2 cδ(n,k )

33 Simulation Setup X(t) = 100 j=1 λj η j V j (t), where λ j = (j 0.5) 2 π 2, V j (t) = 2 sin((j 0.5)πt), η 1,..., η 100 iid N(0, 1). The test function is β B,ξ B 0 = 100 k=1 k 2ξ 1 j=1 j ξ 0.5 V j (t), where B = 0, 0.1, 1 and ξ = 0.1, 0.5, 1. Draw n iid samples from Y = 1 0 X(t)β 0(t)dt + N(0, 1) for n = 100, 500.

34 Figure: Plots of β 0 (t) when B = 1 Setting 1: Plot of β(t) when B=1 β(t) ξ= 0.1 ξ= 0.5 ξ= Time

35 Coverage Proportion of Confidence Interval Table: 100 coverage proportion (average length) of CI when B = ξ = 1 n = 100 n = (0.56) 94.99(0.39)

36 Size Comparison with Hilgert, Mas and Verzelen (2013) Hilgert, Mas and Verzelen (2013) proposed an FPCA-based testing procedure which is adaptive to the truncation parameter k n. We compare our approaches with theirs, denoted HMV. Table: 100 size when B = 0 n = 100 n = 500 HMV PLRT AT

37 Power Comparison with Hilgert, Mas and Verzelen (2013) Table: 100 power when n = 100 Test B = 0.1 B = 1 ξ = 0.1 HMV AT PLRT ξ = 1 HMV AT PLRT

38 Power Comparison with Hilgert, Mas and Verzelen (2013) Table: 100 power when n = 500 Test B = 0.1 B = 1 ξ = 0.1 HMV AT PLRT ξ = 1 HMV AT PLRT

39 Summary We propose applicable procedures for inference in functional data analysis Our approaches do not require perfect alignment Our approaches are asymptotic valid, i.e., desired size and coverage probability The PLRT and adaptive testing procedures are more powerful than existing ones. Extensions to general cases not reported here: quasi-likelihood framework composite hypotheses adaptive testing in non-gaussian error

40 Thank you for your attention!