Karhunen-Loève decomposition of Gaussian measures on Banach spaces Jean-Charles Croix GT APSSE - April 2017, the 13th joint work with Xavier Bay. 1 / 29
Sommaire 1 Preliminaries on Gaussian processes 2 Gaussian measures on Banach spaces Denitions Karhunen-loève decomposition in the Hilbert case Karhunen-loève decomposition on Banach spaces 3 Examples on continuous functions 4 Conclusions 2 / 29
Preliminaries on Gaussian processes Square integrable stochastic process Consider: T a non-empty set, (Ω, F, P) a probability space. A square integrable stochastic process is a collection X = (X t ) t T square integrable real random variables: of t T, X t : (Ω, F, P) (R, B(R)) L 2 (Ω) 3 / 29
Preliminaries on Gaussian processes Square integrable stochastic process Consider: T a non-empty set, (Ω, F, P) a probability space. A square integrable stochastic process is a collection X = (X t ) t T square integrable real random variables: of t T, X t : (Ω, F, P) (R, B(R)) L 2 (Ω) Mean function and covariance kernel Let X be a square integrable process, then t T, m(t) = E [X t ], (t, s) T 2, K(s, t) = E [(X t m(t))(x s m(s))]. From now on, all processes will be centered: m 0. 3 / 29
Preliminaries on Gaussian processes The covariance kernel K is a symmetric kernel: and of positive type: (s, t) T 2, K(s, t) = K(t, s), n N, (t 1,..., t n ) T, (α 1,..., α n ) R n, n α i α j K(t i, t j ) 0. i=1 In short, K is a positive denite kernel on T. Gaussian process A square integrable stochastic process X is Gaussian if and only if: n N, (t 1,..., t n ) T n, (X t1,..., X tn ) is a Gaussian vector. Equivalently, n n µ = α i δ ti, X, µ = α i X ti i=1 i=1 is a real Gaussian variable. 4 / 29
Preliminaries on Gaussian processes Gaussian space Let X be a Gaussian process X on T, the associated Gaussian space is The mapping is a linear and injective. H X = {X t, t T } L2 (Ω,F,P) U : Z H X (t Z, X t L 2) R T Reproducing Kernel Hilbert space (RKHS) Let X be a Gaussian process on T, the associated RKHS is H X = U(H X ). It is a Hilbert space with f, g HX = U 1 (f ), U 1 (g) HX as inner product. 5 / 29
Preliminaries on Gaussian processes The previous mapping U is an isometric isomorphism, thus H K H K as Hilbert spaces. Moore-Aronszajn theorem Let T a set and K a positive-denite kernel on T, then there exists a unique Hilbert subspace H K R T such that: 1 H K = {K(t,.), t T }, 2 t T, h H K, h(t) = h, K t HK. Conversely, let T be a set and K a positive denite symmetric kernel, there exists a Gaussian process on T with K as covariance kernel (Kolmogorov extension theorem). 6 / 29
Preliminaries on Gaussian processes Let T be a set and C T the cylindrical σ-algebra on R T. Any stochastic process X on T is a (C T, F)-measurable mapping X : (Ω, F, P) (R T, C T ). The process X denes a probability measure on (R T, C T ) µ X = P X 1. The dual space ( R ) T = span(δt, t T ) and µ ( R T ), µ X µ 1 is a Gaussian measure on (R, B(R)). This property will be fundamental in the rest of this talk. 7 / 29
Gaussian measures on Banach spaces From now on: (E,. E ) will always be a real (separable) Banach space, B(E) its Borel σ-algebra. Gaussian measures on Banach spaces A probability measure γ on (E, B(E)) is (centered) Gaussian if and only if γ f 1 is a (centered) Gaussian measure on (R, B(R)) for all f E. We will always consider centered Gaussian measure. The characteristic functional is well dened and f E, ˆγ(f ) := e i x,f E,E Cγ (f,f ) γ(dx) = e 2 R with C γ (f, g) = E x, f E,E x, g E,E γ(dx) (covariance). E 8 / 29
Gaussian measures on Banach spaces The covariance is bilinear, symmetric, positive and continuous. Covariance operator Let γ be a Gaussian measure on (E, B(E)) then R γ : f E x, f E,E xγ(dx) E is the covariance operator associated to γ. It is characterized by the following relation (f, g) E E, R γ f, g E,E = C γ (f, g). Moreover, it is a positive symmetric kernel: (f, g) E E R γ f, g E,E = R γ g, f E,E, R γ f, f E,E 0 and R γ is a compact (nuclear) operator. E 9 / 29
Gaussian measures on Banach spaces Similarly to Gaussian processes, dene on R γ (E ) h, g γ = R γ ĥ, ĝ E,E with R γ ĥ = h and R γ ĝ = g. The Cameron-Martin space is H(γ) := and we have the reproducing property {Rγ f, f E } E h H(γ), f E, h, f E,E = h, R γ f γ. (H(γ), <.,. > γ ) is a Hilbertian subspace of E, R γ (X ) is dense in H(γ), R γ may not be injective. 10 / 29
Gaussian measures on Banach spaces As we just saw, R γ : E E need not be injective. However, R γ f = 0 implies f = 0, γ-almost everywhere. Gaussian space Let γ be a Gaussian measure on (E, B(E)) and i : f E f L 2 (γ), the Gaussian space is E γ = i(e ) L2 (γ). For all Gaussian measures, Gaussian and Cameron-Martin spaces are isometric E γ H(γ). 11 / 29
Karhunen-loève decomposition on Hilbert spaces Consider now: (H,.,. H ) a (separable) Hilbert space, B H is weakly compact, H = H (Riesz representation theorem), γ Gaussian measure on (H, B(H)). Continuity h H R γ h, h H is weakly sequentially continuous. As consequence, consider h 0 B H such that: sup R γ h, h H = R γ h 0, h 0 H = λ 0, h H 1 then h 0 is an eigenvector of R γ associated to the eigenvalue λ 0. 12 / 29
Karhunen-loève decomposition on Hilbert spaces The previous h 0 may be used to split H in two orthogonal subspaces From this, dene: P 0 : h H h, h 0 H h 0, γ λ0 = γ P 1 0, γ 1 = γ (I P 0 ) 1. H = Rh 0 H 1. Orthogonal split of the Gaussian measure The two measures γ λ0 and γ 1 on (H, B(H)) are Gaussian and γ = γ λ0 γ 1, R γλ 0 = P 0 R γ P 0 = λ 0 h 0,. H h 0, R γ1 = (I P 0 )R γ (I P 0 ) = R γ R γλ 0 with Rh 0 = H(γ λ0 ), H 1 = (Rh 0 ) both equipped with.,. γ. 13 / 29
Karhunen-loève decomposition on Hilbert spaces Next, apply the splitting method to the (conditional) Gaussian measure γ 1 and so on and so forth... Properties (h n ) is orthonormal in H, (h n ) is dense in H(γ), R γ = n 0 λ n., h n H h n with n 0 λ n <. In particular, R γ is compact (nuclear) and h = n 0 h, h n H h n, γ a.e. The standard theory uses the compacity of R γ as an input and the spectral theorem. Moreover, the class of all Gaussian covariances is equal to symmetric, positive and nuclear operators. 14 / 29
Karhunen-loève decomposition on Banach spaces The following result is already known (Theorem 3.5.1 p.112 in XX) and is called "Karhunen-loève" decomposition in some texts: Decomposition theorem Let γ be a Gaussian measure on a locally convex space E, H(γ) the Cameron-Martin space, (e n ) an orthonormal basis, (ξ n ) n a sequence of independent standard Gaussian random variables on a probability space (Ω, F, P) then ξ n (ω)e n (1) n 0 converges almost surely in E, and has distribution γ. We will now construct a "particular" basis in R γ (E ) which corresponds to decomposition of the covariance operator. 15 / 29
Karhunen-loève decomposition on Banach spaces Consider now the general case: (E,. E ) is a Banach space, B E is weakly compact (Banach alaoglu theorem), γ Gaussian measure on (E, B(E)). Variance continuity f E R γ f, f E,E is weakly sequentially continuous. f 0 B E, Now, write R γ f 0 = λ 0 x 0 E. sup R γ f, f E,E = R γ f 0, f 0 E,E = λ 0. f B X Spectral theory is not dened in this context. 16 / 29
Karhunen-loève decomposition on Banach spaces (f 0, x 0 ) will be useful to split "orthogonally" the Banach space and similarly, dene P 0 : x E x, f 0 E,E x 0, γ λ0 = γ P 1 0, γ 1 = γ (I P 0 ) 1. E = Rx 0 + X 1, Orthogonal split of the Gaussian measure The two measures γ λ0 and γ 1 on (E, B(E)) are Gaussian and γ = γ λ0 γ 1, R γλ 0 = P 0 R γ P 0 = λ 0 h 0,. H h 0, R γ1 = (I P 0 )R γ (I P 0 ) = R γ R γλ 0 with Rh 0 = H(γ λ0 ), H 1 = (Rh 0 ) both equipped with.,. γ. 17 / 29
Karhunen-loève decomposition on Banach spaces Again, repeat the splitting method on γ 1. Properties x n E = f n E = 1, (x n ) is dense in H(γ), R γ = λ n 0 n., f n E,E x n. Again, we have: x = x, fn E,E x n, γ a.e. n 0 and R γ = n 0 λ n x n,. E,E x n. The class of Gaussian covariance operators is still an open problem. 18 / 29
Consequences So far, we have the following remarks: Approximation of covariance operator In the previous decomposition, we have: n R γ λ k x k,. E,E x k = λ n+1 k=0 where the norm is in L(X, X ). Maximum variance sup R γn f, f E,E = sup Var( P n X, f E,E ) = λ n f B E f B E λ n 0 n may not be nite, The decomposition may not be unique. 19 / 29
Specify a Gaussian measure in practice Given a function space E, to specify a Gaussian measure on a function space: choose a covariance kernel K such that: P [(t X t ) E] = 1 choose directly a Covariance operator (ex: inverse of elliptic dierential operator) Sample path from stochastic processes have properties linked to the Kernel: Continuity (Kolmogorov continuity theorem), Integrability, Dierentiability... 20 / 29
Example on the Wiener measure Now, consider the standard Brownian motion with kernel K(s, t) = s t. On H = L 2 ([0, 1], R), the decomposition is: h n : t (( 2 sin n + π ) ) t, 2 1 λ n = ( ) n + π 2 2 On E = C([0, 1], R), it becomes: h n : t 2 (1 p { 2k 2k+1 2p+1, 2 } 1 { p+1 2k+1 2 2k+2 p+1, 2 } p+1 ), x n : t 2 p+2 h n(t), λ n = 1 2, n = p+2 2p + k, k = 0,..., 2 p 1, p 0 which represent "hat functions" on dyadic intervals. 21 / 29
Example with dierent kernels Basis functions 1.2 x0 x1 x2 x3 1.0 x4 x5 x6 x7 x8 0.8 x9 x10 x11 0.6 0.4 0.2 0.0 0.2 0.0 0.2 0.4 0.6 0.8 1.0 Figure: Wiener measure basis functions. 22 / 29
Example with dierent kernels 10 1 Variances Variances 10 0 10-1 10-2 0 2 4 6 8 10 Figure: Wiener measure basis variances. 23 / 29
Example with dierent kernels 1.5 Basis functions 1.0 0.5 0.0 x0 x1 x2 0.5 x3 x4 x5 x6 x7 x8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Figure: Gaussian kernel basis functions. 24 / 29
Example with dierent kernels 10 1 Variances Variances 10 0 10-1 10-2 10-3 10-4 10-5 10-6 10-7 10-8 10-9 10-10 10-11 10-12 10-13 0 1 2 3 4 5 6 7 8 Figure: Gaussian kernel basis variances. 25 / 29
Example with dierent kernels Basis functions x0 x1 1.0 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 0.5 0.0 0.5 0.0 0.2 0.4 0.6 0.8 1.0 Figure: Matern 3/2 kernel basis functions. 26 / 29
Example with dierent kernels 10 1 Variances Variances 10 0 10-1 10-2 10-3 10-4 0 2 4 6 8 10 Figure: Matern 3/2 kernel basis variances. 27 / 29
Brownian motion simulation 0.5 Brownian motion simulation Sin basis with n=1e4 Sin basis with n=1e2 Schauder basis with n=64 0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.2 0.4 0.6 0.8 1.0 Figure: Simulation of brownian motion using dierent bases. 28 / 29
Conclusion A new method to decompose Gaussian measures on Banach spaces. The method can be applied to non-gaussian square integrable measures, Gaussian covariance characterization is still an open problem, Links to optimal design in Kriging, Links with Bayesian inverse problems, Precise quantication of uncertainty. 29 / 29