Hypothesis testing for Stochastic PDEs Igor Cialenco Department of Applied Mathematics Illinois Institute of Technology igor@math.iit.edu Joint work with Liaosha Xu Research partially funded by NSF grants DMS-1211256 Asymptotical Statistics of Stochastic Processes X Le Mans, 17-20 March, 2015
Outline of the talk Overview of Parameter Estimation for SPDEs Hypothesis testing Error estimation Numerical results Ig. Cialenco, IIT March 17-20, 2015 Slide # 2
Bibliography I. Cialenco, L. Xu, Hypothesis testing for stochastic PDEs driven by additive noise, Stochastic Processes and their Applications, vol. 125, Issue 3, March 2015, pp. 819-866. I. Cialenco, L. Xu, A note on error estimation for hypothesis testing problems for some linear SPDEs, Stochastic Partial Differential Equations: Analysis and Computations, September 2014, vol. 2, No 3, pp. 408-431. Ig. Cialenco, IIT March 17-20, 2015 Slide # 3
The Setup Let (Ω, F, {F t } t 0, P) be a stochastic basis with usual assumptions. Consider heat equation driven by additive noise: du(t, x) θ U(t, x)dt = σ k=1 where x G, G is a bounded domain in R d, t [0, T ]; λ γ k h k(x)dw k (t), (0.1) θ > 0 is the parameter of interest; γ 0, σ R {0} are known. zero initial conditions and boundary values; {w k (t)} k N are independent Brownian motions; is the Laplace operator on G with zero boundary condition U = d j=1 2 U ; x 2 j {h k } are the eigenfunctions of in L 2 (G), and {ρ k } are the corresponding eigenvalues; λ k = ρ k k 1/d ; Ig. Cialenco, IIT March 17-20, 2015 Slide # 4
Existence of the solution Theorem (Existence and Uniqueness) If (γ s)/d > 1/2, then the SPDE (0.1) has a unique solution (weak in PDE sense, and strong in probability sense) U L 2 (Ω [0, T ]; H s+1 (G)) L 2 (Ω; C((0, T ); H s (G))). Ig. Cialenco, IIT March 17-20, 2015 Slide # 5
Construction of the test Part I: Hypothesis Testing for SPDEs Ig. Cialenco, IIT March 17-20, 2015 Slide # 6
The Problem: Simple Hypothesis Hypothesis Testing Construction of the test du(t, x) θ U(t, x)dt = σ k=1 Assume that θ can take only two values {, θ 1 }. Consider a simple hypothesis: H 0 θ =, H 1 θ = θ 1. For simplicity, assume θ 1 > and σ > 0. λ γ k h k(x)dw k (t). Ig. Cialenco, IIT March 17-20, 2015 Slide # 7
Construction of the Test Hypothesis Testing Construction of the test du(t, x) θ U(t, x)dt = σ k=1 λ γ k h k(x)dw k (t), U(0, x) = 0. The k-th Fourier coefficient u k (t) = U(t, x), h k (x) is given by du k = θλ 2 k u kdt + σλ γ k dw k(t), u k (0) = 0, u k (t) = σλ γ k t 0 e θλ2 k (t s) dw k, k 1. Let P N,T θ ( ) = P(U N T ) be the measure on C([0, T ]; RN ) generated by U N T (t) = (u 1,..., u N ) up to time T. Ig. Cialenco, IIT March 17-20, 2015 Slide # 8
Construction of the test Observable: First N Fourier coefficients, for all t [0, T ], U N T = ((u 1 (t),..., u N (t)), t [0, T ]). In this case, the Likelihood Ratio has the form L(, θ 1, UT N ) = PN,T θ 1 P N,T θ 1 ( 0 = exp ( (θ 1 )σ 2 T N λ 2+2γ k k=1 u k (t)du k (t) + 1 2 (θ 1 + )λ 2 k and the Maximum Likelihood Estimator is 0 T u 2 k (t)dt)), θ N T = N k=1 λ2+2γ k N k=1 λ4+2γ k T 0 u k(t)du k (t) T 0 u2 k (t)dt, N N, T > 0. Ig. Cialenco, IIT March 17-20, 2015 Slide # 9
Construction of the test Two Asymptotic Regimes: Large times, T ; Large number of Fourier modes, N. Theorem Assume that 2γ > d. Then, lim θ T N N = lim T θn T = θ, a.e. and where M = N k=1 λ2 k. (4/d + 2)θ ϖσ 2 T ). lim N 1/d+ 1 2 ( θ T N θ) = d N (0, N T ( θn T θ) = d N (0, 2θ/M), lim T Ig. Cialenco, IIT March 17-20, 2015 Slide # 10
Construction of the test Looking for rejection region R B(C([0, T ]; R N )). Type I error = P N,T (R); Type II error = 1 P N,T θ 1 Define the class of test (R), and power of the test = P N,T θ 1 (R) K α = {R B(C([0, T ]; R N )) P N,T (R) α}. with α (0, 1) being the significant level, fixed in what follows. Definition We say that a rejection region R K α is the most powerful in the class K α if P N,T θ 1 (R) P N,T θ 1 (R ), for all R K α. Ig. Cialenco, IIT March 17-20, 2015 Slide # 11
Neyman-Pearson Lemma Hypothesis Testing Most Powerful Test Theorem (C. and Xu, 14) Let c α be a real number such that P N,T (L(, θ 1, U N T ) c α ) = α. Then, R = {U N T L(, θ 1, U N T ) c α }, is the most powerful rejection region in the class K α. Ig. Cialenco, IIT March 17-20, 2015 Slide # 12
The Difficulty: c α has no explicit formula for finite T and N. Ig. Cialenco, IIT March 17-20, 2015 Slide # 13
The Difficulty: c α has no explicit formula for finite T and N. We take/suggest an Asymptotic Method (1) Fix N, let T ; (2) Fix T, let N. Inspired by large times Asymptotic Method for finite dimensional ergodic diffusion processes developed by Kutoyants 74, and 04. In this talk we focus on case (2), asymptotics in number of Fourier modes; For case (1) see [CX 14]. Ig. Cialenco, IIT March 17-20, 2015 Slide # 13
umber of Fourier modes N Define a new class K α = {(R N ) N N R N B(C([0, T ]; R N ), lim sup P N,T (R N ) α}, N where T is fixed, and α is the Asymptotic Significant Level. Ig. Cialenco, IIT March 17-20, 2015 Slide # 14
umber of Fourier modes N Define a new class K α = {(R N ) N N R N B(C([0, T ]; R N ), lim sup P N,T (R N ) α}, N where T is fixed, and α is the Asymptotic Significant Level. Goal: We want to find a rejection region ( R N ) N N such that lim N P N,T ( R N ) = α. Ig. Cialenco, IIT March 17-20, 2015 Slide # 14
Attempt: We still try Likelihood Ratio test. Then, what is c α? P N,T = P N,T where (L(, θ 1, U N T ) c α (N)) XN Y N X N = 8θ 3 0 ln c α (N) T M(θ 2 1 θ0 2) + 2θ0 T M(θ 1 ) 2(θ 1 + ) N θ0 (θ 1 ) σ 2 2T M(θ 1 + ) ( k=1 2θ0 T Y N = σ T M M = N k=1 λ 2 k. N λ 2+γ k k=1 0 λ 2+2γ k u 2 σ2 k (T ) N), 2 u k dw k, (θ 1 )N 8θ0 T M(θ 1 + ), Ig. Cialenco, IIT March 17-20, 2015 Slide # 15
For δ R, by splitting argument, we get P N,T + P N,T (L(, θ 1, UT N ) c α (N)) P N,T (X N δ) Y N 8θ 3 0 ln c α (N) T M(θ 2 1 θ0 2) + 2θ0 T M(θ 1 ) 2(θ 1 + ) (θ 1 )N 8θ0 T M(θ 1 + ) δ. Take δ = N 1/2d, and deduce that P N,T (X N δ) 0, N. Use the fact that Y N d N (0, 1) as N, and find Reasonable candidate for the threshold constant ĉ α (N) = exp (θ 1 ) 2 T M 4 + (θ 1 ) 2 N 8θ 2 0 T M(θ 2 1 θ0 2 ) q α 8θ 3 0. Ig. Cialenco, IIT March 17-20, 2015 Slide # 16
Theorem (C. and Xu) Suppose R N = {U N T L(, θ 1, U N T ) ĉ α (N)}, for all N, where ĉ α is given above. Then, the rejection region ( R N ) N N K α, and moreover lim N PN,T ( R N ) = α. Ig. Cialenco, IIT March 17-20, 2015 Slide # 17
The Next Question: Is ( R N ) N N (asymptotically) the most powerful test in K α? Ig. Cialenco, IIT March 17-20, 2015 Slide # 18
The Next Question: Is ( R N ) N N (asymptotically) the most powerful test in K α? Short answer: No! Ig. Cialenco, IIT March 17-20, 2015 Slide # 18
The Next Question: Is ( R N ) N N (asymptotically) the most powerful test in K α? Short answer: No! How does the power of this test P N,T θ 1 ( R N ) behave? Definition We say that a rejection region (R N ) K α is asymptotically the most powerful in the class K α if 1 P N,T θ lim inf 1 (R N ) N 1 P N,T θ 1 (RN ) 1, for all (R N) Kα. Similarly, we define asymptotically the most powerful rejection regions for a different given class of tests. Ig. Cialenco, IIT March 17-20, 2015 Slide # 18
Theorem (C. and Xu) (Criterion for Most Powerful Test) Consider the rejection region of the form R N = {U N T L(, θ 1, U N T ) c α(n)}, where c α(n) is a function of N such that, c α(n) > 0 for all N N, and lim N PN,T (R N) = α, lim N c α(n) 1 P N,T θ 1 (R N ) <. Then (R N ) is asymptotically the most powerful in K α. Is the test R N asymptotically the most powerful in the class K α? We need to control the power of the test P N,T θ 1 ( R N ), as N. Ig. Cialenco, IIT March 17-20, 2015 Slide # 19
Calculate the Cumulant Generating Function of the Log-Likelihood ratio m(ɛ) = E[exp(ɛ ln L(, θ 1, U N T ))] m(ɛ) = exp [ 1 2 N k=1 ln (cosh(γ k T ) p sinh(γ k T )) + Similar to Gapeev and Küchler [2008] Use Feynman-Kac Formula to derive a PDE Make some transforms and guess the solution ɛ(θ1 θ0) + θ1 MT ]. 2 Sharp Large Deviations (see Lin kov 1999, and Bercu and Roualt 2001 for T ) L T (ɛ) = T 1 ln E θ1 [exp (ɛ ln L(, θ 1, U N T ))] = L(ɛ) + T 1 H(ɛ) + T 1 R T (ɛ), L N (ɛ) = M 1 ln E θ1 [exp (ɛ ln L(, θ 1, U N T ))] = L(ɛ) + NM 1 H(ɛ) + M 1 RN (ɛ), N M = λ 2 k N 2/d+1. k=1 Ig. Cialenco, IIT March 17-20, 2015 Slide # 20
Corollary After a series of technical results, we get ĉ α (N)/ (1 P N,T θ 1 ( R N )) M N 1/d+1/2, and hence R N is NOT assymptoticaly most powerful in the class K α. Ig. Cialenco, IIT March 17-20, 2015 Slide # 21
Corollary After a series of technical results, we get ĉ α (N)/ (1 P N,T θ 1 ( R N )) M N 1/d+1/2, and hence R N is NOT assymptoticaly most powerful in the class K α. Solution: a new class of tests Similar remark is true for T : c α(t )/ (1 P N,T θ 1 (R T )) T, as T, where R T ={U N T L(, θ 1, U N T ) c α(t )}, for all T, c α(t ) = exp (θ1 θ0)2 MT θ2 1 θ0 2 MT q α 4 2 2. Ig. Cialenco, IIT March 17-20, 2015 Slide # 21
Control of Type I error Hypothesis Testing Proposition (C. and Xu) For any x R, we have the following expansion, P N,T (I N x) = Φ(x) + Φ δ 1(x)M 1/2 + Φ δ 2(x)NM 1 + R δ N(x) (M 1 + NM 3/2 + N 2 M 2 ), where I N = 8θ 3 0 ln L(,θ 1,U N T ) T M(θ 2 1 θ 2 0 ) 2θ0 T M(θ 1 ) 2(θ 1 + ) + (θ 1 )N 8θ0 T M(θ 1 + ), and Φ( ) is the distribution of a standard Gaussian random variable, and Φ 1 ( ) and Φ 2 ( ) are bounded smooth functions and all their derivatives are bounded, and R N ( ) is uniformly bounded in x. Ig. Cialenco, IIT March 17-20, 2015 Slide # 22
Main result Hypothesis Testing Define ˆK α = {(R N ) lim sup (P N,T (R N ) α) M α 1 }. N Note that R N = {U N T IN q α }, so by the previous lemma we have where α 1 is an explicit constant. lim N (PN,T ( ˆR N ) α) M = ˆα 1, Ig. Cialenco, IIT March 17-20, 2015 Slide # 23
Main result Hypothesis Testing Define ˆK α = {(R N ) lim sup (P N,T (R N ) α) M α 1 }. N Note that R N = {U N T IN q α }, so by the previous lemma we have where α 1 is an explicit constant. Theorem (C. and Xu) lim N (PN,T ( ˆR N ) α) M = ˆα 1, The rejection region ( ˆR N ) is asymptotically the most powerful in ˆK α. Ig. Cialenco, IIT March 17-20, 2015 Slide # 23
Part II: Error Control and Numerical Results Ig. Cialenco, IIT March 17-20, 2015 Slide # 24
New Tests for Error Control, N Hypothesis Testing New Tests for Error Control Theorem (C. and Xu) Consider the test statistics of the form R 0 N = {U N T ln L(, θ 1, U N T ) ζ 0 M}, where ζ 0 is given by an explicit formula of the form (θ 1 ) 2 4 T + O(N 1/2 1/d ). If N N 0 (N 0 has explicit formula), then the Type I and Type II errors have the following bound estimates P N,T (RN 0 ) (1 + ϱ)α, 1 P N,T θ 1 (RN 0 ) (1 + ϱ) exp ( (θ 1 ) 2 16θ0 2 MT ), where ϱ denotes a given threshold of error tolerance. Ig. Cialenco, IIT March 17-20, 2015 Slide # 25
Table: Type I errors for various N N 10 20 30 40 50 60 70 80 P N,T (RN 0 ) 0.007 0.012 0.010 0.017 0.012 0.014 0.010 0.013 P N,T (RN ) 0.006 0.037 0.039 0.053 0.040 0.039 0.054 0.046 Other parameters: α = 0.05, = 0.1, θ 1 = 0.2, T = 1, ϱ = 0.1, d = σ = 1, γ = 0
Table: T = Tb 1 given by theoretical results and Type I error for various α. α 0.1 0.05 0.01 0.005 T 1 b 629 818 1258 1447 P N,T (RT 0 ) 0.021 0.010 0.0025 0.0015 Other parameters: = 0.1, θ 1 = 0.2, N = 3, ϱ = 0.1, d = σ = 1, γ = 0
Table: Type II errors for various T. T 10 20 30 40 50 60 exp ( (θ 1 ) 2 16 MT ) 0.4169 0.1738 0.0724 0.0302 0.0126 0.0052 1 P N,T θ 1 (R 0 T ) 0.7155 0.3329 0.1148 0.0293 0.0070 0.0012 1 P N,T θ 1 (R T ) 0.7946 0.2402 0.0457 0.0060 0.0006 0.0002 Other parameters: α = 0.05, = 0.1, θ 1 = 0.2, N = 3, ϱ = 0.1, d = σ = 1, γ = 0
Thank You! The end of the talk... but not of the story