Karhunen-Loève decomposition of Gaussian measures on Banach spaces

Transcription

1 Karhunen-Loève decomposition of Gaussian measures on Banach spaces Jean-Charles Croix GT APSSE - April 2017, the 13th joint work with Xavier Bay. 1 / 29

2 Sommaire 1 Preliminaries on Gaussian processes 2 Gaussian measures on Banach spaces Denitions Karhunen-loève decomposition in the Hilbert case Karhunen-loève decomposition on Banach spaces 3 Examples on continuous functions 4 Conclusions 2 / 29

3 Preliminaries on Gaussian processes Square integrable stochastic process Consider: T a non-empty set, (Ω, F, P) a probability space. A square integrable stochastic process is a collection X = (X t ) t T square integrable real random variables: of t T, X t : (Ω, F, P) (R, B(R)) L 2 (Ω) 3 / 29

4 Preliminaries on Gaussian processes Square integrable stochastic process Consider: T a non-empty set, (Ω, F, P) a probability space. A square integrable stochastic process is a collection X = (X t ) t T square integrable real random variables: of t T, X t : (Ω, F, P) (R, B(R)) L 2 (Ω) Mean function and covariance kernel Let X be a square integrable process, then t T, m(t) = E [X t ], (t, s) T 2, K(s, t) = E [(X t m(t))(x s m(s))]. From now on, all processes will be centered: m 0. 3 / 29

5 Preliminaries on Gaussian processes The covariance kernel K is a symmetric kernel: and of positive type: (s, t) T 2, K(s, t) = K(t, s), n N, (t 1,..., t n ) T, (α 1,..., α n ) R n, n α i α j K(t i, t j ) 0. i=1 In short, K is a positive denite kernel on T. Gaussian process A square integrable stochastic process X is Gaussian if and only if: n N, (t 1,..., t n ) T n, (X t1,..., X tn ) is a Gaussian vector. Equivalently, n n µ = α i δ ti, X, µ = α i X ti i=1 i=1 is a real Gaussian variable. 4 / 29

6 Preliminaries on Gaussian processes Gaussian space Let X be a Gaussian process X on T, the associated Gaussian space is The mapping is a linear and injective. H X = {X t, t T } L2 (Ω,F,P) U : Z H X (t Z, X t L 2) R T Reproducing Kernel Hilbert space (RKHS) Let X be a Gaussian process on T, the associated RKHS is H X = U(H X ). It is a Hilbert space with f, g HX = U 1 (f ), U 1 (g) HX as inner product. 5 / 29

7 Preliminaries on Gaussian processes The previous mapping U is an isometric isomorphism, thus H K H K as Hilbert spaces. Moore-Aronszajn theorem Let T a set and K a positive-denite kernel on T, then there exists a unique Hilbert subspace H K R T such that: 1 H K = {K(t,.), t T }, 2 t T, h H K, h(t) = h, K t HK. Conversely, let T be a set and K a positive denite symmetric kernel, there exists a Gaussian process on T with K as covariance kernel (Kolmogorov extension theorem). 6 / 29

8 Preliminaries on Gaussian processes Let T be a set and C T the cylindrical σ-algebra on R T. Any stochastic process X on T is a (C T, F)-measurable mapping X : (Ω, F, P) (R T, C T ). The process X denes a probability measure on (R T, C T ) µ X = P X 1. The dual space ( R ) T = span(δt, t T ) and µ ( R T ), µ X µ 1 is a Gaussian measure on (R, B(R)). This property will be fundamental in the rest of this talk. 7 / 29

9 Gaussian measures on Banach spaces From now on: (E,. E ) will always be a real (separable) Banach space, B(E) its Borel σ-algebra. Gaussian measures on Banach spaces A probability measure γ on (E, B(E)) is (centered) Gaussian if and only if γ f 1 is a (centered) Gaussian measure on (R, B(R)) for all f E. We will always consider centered Gaussian measure. The characteristic functional is well dened and f E, ˆγ(f ) := e i x,f E,E Cγ (f,f ) γ(dx) = e 2 R with C γ (f, g) = E x, f E,E x, g E,E γ(dx) (covariance). E 8 / 29

10 Gaussian measures on Banach spaces The covariance is bilinear, symmetric, positive and continuous. Covariance operator Let γ be a Gaussian measure on (E, B(E)) then R γ : f E x, f E,E xγ(dx) E is the covariance operator associated to γ. It is characterized by the following relation (f, g) E E, R γ f, g E,E = C γ (f, g). Moreover, it is a positive symmetric kernel: (f, g) E E R γ f, g E,E = R γ g, f E,E, R γ f, f E,E 0 and R γ is a compact (nuclear) operator. E 9 / 29

11 Gaussian measures on Banach spaces Similarly to Gaussian processes, dene on R γ (E ) h, g γ = R γ ĥ, ĝ E,E with R γ ĥ = h and R γ ĝ = g. The Cameron-Martin space is H(γ) := and we have the reproducing property {Rγ f, f E } E h H(γ), f E, h, f E,E = h, R γ f γ. (H(γ), <.,. > γ ) is a Hilbertian subspace of E, R γ (X ) is dense in H(γ), R γ may not be injective. 10 / 29

12 Gaussian measures on Banach spaces As we just saw, R γ : E E need not be injective. However, R γ f = 0 implies f = 0, γ-almost everywhere. Gaussian space Let γ be a Gaussian measure on (E, B(E)) and i : f E f L 2 (γ), the Gaussian space is E γ = i(e ) L2 (γ). For all Gaussian measures, Gaussian and Cameron-Martin spaces are isometric E γ H(γ). 11 / 29

13 Karhunen-loève decomposition on Hilbert spaces Consider now: (H,.,. H ) a (separable) Hilbert space, B H is weakly compact, H = H (Riesz representation theorem), γ Gaussian measure on (H, B(H)). Continuity h H R γ h, h H is weakly sequentially continuous. As consequence, consider h 0 B H such that: sup R γ h, h H = R γ h 0, h 0 H = λ 0, h H 1 then h 0 is an eigenvector of R γ associated to the eigenvalue λ / 29

14 Karhunen-loève decomposition on Hilbert spaces The previous h 0 may be used to split H in two orthogonal subspaces From this, dene: P 0 : h H h, h 0 H h 0, γ λ0 = γ P 1 0, γ 1 = γ (I P 0 ) 1. H = Rh 0 H 1. Orthogonal split of the Gaussian measure The two measures γ λ0 and γ 1 on (H, B(H)) are Gaussian and γ = γ λ0 γ 1, R γλ 0 = P 0 R γ P 0 = λ 0 h 0,. H h 0, R γ1 = (I P 0 )R γ (I P 0 ) = R γ R γλ 0 with Rh 0 = H(γ λ0 ), H 1 = (Rh 0 ) both equipped with.,. γ. 13 / 29

15 Karhunen-loève decomposition on Hilbert spaces Next, apply the splitting method to the (conditional) Gaussian measure γ 1 and so on and so forth... Properties (h n ) is orthonormal in H, (h n ) is dense in H(γ), R γ = n 0 λ n., h n H h n with n 0 λ n <. In particular, R γ is compact (nuclear) and h = n 0 h, h n H h n, γ a.e. The standard theory uses the compacity of R γ as an input and the spectral theorem. Moreover, the class of all Gaussian covariances is equal to symmetric, positive and nuclear operators. 14 / 29

16 Karhunen-loève decomposition on Banach spaces The following result is already known (Theorem p.112 in XX) and is called "Karhunen-loève" decomposition in some texts: Decomposition theorem Let γ be a Gaussian measure on a locally convex space E, H(γ) the Cameron-Martin space, (e n ) an orthonormal basis, (ξ n ) n a sequence of independent standard Gaussian random variables on a probability space (Ω, F, P) then ξ n (ω)e n (1) n 0 converges almost surely in E, and has distribution γ. We will now construct a "particular" basis in R γ (E ) which corresponds to decomposition of the covariance operator. 15 / 29

17 Karhunen-loève decomposition on Banach spaces Consider now the general case: (E,. E ) is a Banach space, B E is weakly compact (Banach alaoglu theorem), γ Gaussian measure on (E, B(E)). Variance continuity f E R γ f, f E,E is weakly sequentially continuous. f 0 B E, Now, write R γ f 0 = λ 0 x 0 E. sup R γ f, f E,E = R γ f 0, f 0 E,E = λ 0. f B X Spectral theory is not dened in this context. 16 / 29

18 Karhunen-loève decomposition on Banach spaces (f 0, x 0 ) will be useful to split "orthogonally" the Banach space and similarly, dene P 0 : x E x, f 0 E,E x 0, γ λ0 = γ P 1 0, γ 1 = γ (I P 0 ) 1. E = Rx 0 + X 1, Orthogonal split of the Gaussian measure The two measures γ λ0 and γ 1 on (E, B(E)) are Gaussian and γ = γ λ0 γ 1, R γλ 0 = P 0 R γ P 0 = λ 0 h 0,. H h 0, R γ1 = (I P 0 )R γ (I P 0 ) = R γ R γλ 0 with Rh 0 = H(γ λ0 ), H 1 = (Rh 0 ) both equipped with.,. γ. 17 / 29

19 Karhunen-loève decomposition on Banach spaces Again, repeat the splitting method on γ 1. Properties x n E = f n E = 1, (x n ) is dense in H(γ), R γ = λ n 0 n., f n E,E x n. Again, we have: x = x, fn E,E x n, γ a.e. n 0 and R γ = n 0 λ n x n,. E,E x n. The class of Gaussian covariance operators is still an open problem. 18 / 29

20 Consequences So far, we have the following remarks: Approximation of covariance operator In the previous decomposition, we have: n R γ λ k x k,. E,E x k = λ n+1 k=0 where the norm is in L(X, X ). Maximum variance sup R γn f, f E,E = sup Var( P n X, f E,E ) = λ n f B E f B E λ n 0 n may not be nite, The decomposition may not be unique. 19 / 29

21 Specify a Gaussian measure in practice Given a function space E, to specify a Gaussian measure on a function space: choose a covariance kernel K such that: P [(t X t ) E] = 1 choose directly a Covariance operator (ex: inverse of elliptic dierential operator) Sample path from stochastic processes have properties linked to the Kernel: Continuity (Kolmogorov continuity theorem), Integrability, Dierentiability / 29

22 Example on the Wiener measure Now, consider the standard Brownian motion with kernel K(s, t) = s t. On H = L 2 ([0, 1], R), the decomposition is: h n : t (( 2 sin n + π ) ) t, 2 1 λ n = ( ) n + π 2 2 On E = C([0, 1], R), it becomes: h n : t 2 (1 p { 2k 2k+1 2p+1, 2 } 1 { p+1 2k+1 2 2k+2 p+1, 2 } p+1 ), x n : t 2 p+2 h n(t), λ n = 1 2, n = p+2 2p + k, k = 0,..., 2 p 1, p 0 which represent "hat functions" on dyadic intervals. 21 / 29

23 Example with dierent kernels Basis functions 1.2 x0 x1 x2 x3 1.0 x4 x5 x6 x7 x8 0.8 x9 x10 x Figure: Wiener measure basis functions. 22 / 29

24 Example with dierent kernels 10 1 Variances Variances Figure: Wiener measure basis variances. 23 / 29

25 Example with dierent kernels 1.5 Basis functions x0 x1 x2 0.5 x3 x4 x5 x6 x7 x Figure: Gaussian kernel basis functions. 24 / 29

26 Example with dierent kernels 10 1 Variances Variances Figure: Gaussian kernel basis variances. 25 / 29

27 Example with dierent kernels Basis functions x0 x1 1.0 x2 x3 x4 x5 x6 x7 x8 x9 x10 x Figure: Matern 3/2 kernel basis functions. 26 / 29

28 Example with dierent kernels 10 1 Variances Variances Figure: Matern 3/2 kernel basis variances. 27 / 29

29 Brownian motion simulation 0.5 Brownian motion simulation Sin basis with n=1e4 Sin basis with n=1e2 Schauder basis with n= Figure: Simulation of brownian motion using dierent bases. 28 / 29

30 Conclusion A new method to decompose Gaussian measures on Banach spaces. The method can be applied to non-gaussian square integrable measures, Gaussian covariance characterization is still an open problem, Links to optimal design in Kriging, Links with Bayesian inverse problems, Precise quantication of uncertainty. 29 / 29