tifica Lecture 1: Computation of Karhunen-Loeve Expansion: Alexander Litvinenko http://sri-uq.kaust.edu.sa/
Stochastic PDEs We consider div(κ(x, ω) u) = f (x, ω) in G, u = 0 on G, with stochastic coefficients κ(x, ω), x G R d and ω belongs to the space of random events Ω. Methods and techniques: Response surface, Monte-Carlo, Perturbation, Stochastic Galerkin. 2 / 25
Covariance functions The random field requires to specify its spatial correl. structure cov f (x, y) = E[(f (x, ) µ f (x))(f (y, ) µ f (y))], 3 Let h = i=1 h2 i /l2 i, where h i := x i y i, i = 1, 2, 3, l i are cov. lengths. Examples: Gaussian cov(h) = exp( h 2 ), exponential cov(h) = exp( h), 3 / 25
/ 25 Figure : Gaussian cov(x, y) = exp( 10 x y 2 ) and exponential cov(x, y) = exp( 10 x y ) covariance functions.
KLE The spectral representation of the cov. function is C κ (x, y) = i=0 λ ik i (x)k i (y), where λ i and k i (x) are the eigenvalues and eigenfunctions. The Karhunen-Loève expansion is the series κ(x, ω) = µ k (x) + λi k i (x)ξ i (ω), i=1 where ξ i (ω) are uncorrelated random variables and k i are basis functions in L 2 (G). Eigenpairs λ i, k i are the solution of Tk i = λ i k i, k i L 2 (G), i N, where. T : L 2 (G) L 2 (G), (Tu)(x) := G cov k(x, y)u(y)dy. 5 / 25
Computation of eigenpairs by FFT If the cov. function depends on (x y) then on a uniform tensor grid the cov. matrix C is (block) Toeplitz. Then C can be extended to the (block) circulant one and C = 1 n F H ΛF (1) may be computed like follows. Multiply (1) by F, obtain F C = ΛF, for the first column we have F C 1 = ΛF 1. Since all entries of F 1 are unity, obtain λ = F C 1. F C 1 may be computed very efficiently by FFT [Cooley, 1965] in O(n log n) FLOPS. C 1 may be represented in a matrix or in a tensor format. 6 / 25
Multidimensional FFT Lemma: Let U R n 1... n d be a given tensor, n := n 1 n 2... n d and vector u R n contains all elements of U columnwise. Let R n i n i, i = 1,..., d. Then for the d-dim. FT holds F (1) i ũ = F (1) d... F (1) 2 F (1) 1 u, where vector ũ R n contains columnwise all elements of d-dim. FT of U R n n. Lemma: Let C R n m and C = k i=0 a ibi T, where a i R n, b i R m. Then F (2) (C) = k F (1) (a i )F (1) (bi T ). (2) i=0 7 / 25
MATLAB code: Algorithm Computing 2-dimensional DFT matrix F (2) begin F (1) 1 =dftmtx(n 1); F (1) 2 =dftmtx(n 2); F (2) =kron(f 2, F 1 ); U=rand(n 1, n 2 ); norm(fftn(u)-reshape(f (2) U(:), n 1, n 2 ), inf ); end; Limits 85 2 dofs in R 2 or 18 3 dofs in R 3. 8 / 25
Discrete eigenvalue problem Let W ij := k,m G b i (x)b k (x)dxc km M ij = G G b i (x)b j (x)dx. b j (y)b m (y)dy, Then we solve W fl h = λ lmfl h, where W := MCM Approximate C and M in the H-matrix format tion Logo low Lock-up Kronecker rank format and use the Lanczos method to compute m largest eigenvalues. 9 / 25
10 / 25 Table : Time required for computing m eigenpairs, cov(h) = exp( h), l 1 = l 2 = 0.1. The time to compute the H-matrix approximation is 26 sec. and the storage requirement for C H is 1.17 GB. m 10 20 0 80 160 total time (sec.), l 1 = l 2 = 1 7 16 3 109 total time (sec.), l 1 = l 2 = 0.1 35 51 97 19 532
25 20 20 20 20 20 16 16 20 20 16 20 16 20 16 16 16 16 20 16 16 20 20 20 20 32 20 32 3 20 16 20 20 20 20 32 20 32 20 20 20 20 20 32 20 32 20 20 20 20 20 32 20 20 20 20 20 32 20 32 20 20 20 20 20 32 25 11 20 11 11 20 20 8 10 20 32 20 11 11 20 20 10 10 20 12 32 32 20 11 11 20 20 9 9 20 32 32 32 32 32 Examples of H-matrix approximates of cov(x, y) = e 2 x y 20 32 3 11 20 12 20 11 9 16 32 32 32 32 12 32 32 32 32 32 32 32 32 32 32 32 Figure : H-matrix approximations R n n, n = 32 2, with standard (left) and weak (right) admissibility block partitionings. The biggest dense (dark) blocks R n n, max. rank k = left and k = right. 32 11 / 25
H - Matrices To assemble low-rank blocks use ACA. Dependence of the computational time and storage requirements of C H on the rank k, n = 32 2. k time (sec.) memory (MB) C C H 2 C 2 2 0.0 2 3.5e 5 6 0.1 1.e 5 9 0.1 5. 1.e 5 12 0.17 6.8 3.1e 7 17 0.23 9.3 6.3e 8 The time for dense matrix C is 3.3 sec. and the storage 10 MB. 12 / 25
H - Matrices exponential cov(h) = exp( h), The cov. matrix C R n n, n = 65 2. l 1 l 2 C C H 2 C 2 0.01 0.02 3e 2 0.1 0.2 8e 3 1 2 2.8e 6 / 25
700 600 500 00 300 200 100 1800 1600 100 1200 1000 800 600 00 200 150 100 50 0 0 100 200 300 00 500 600 700 800 900 1000 0 0 100 200 300 00 500 600 700 800 900 1000 0 0 100 200 300 00 500 600 700 800 900 1000 9 8 7 6 5 3 2 1 1.5 0.5 0 0 100 200 300 00 500 600 700 800 900 1000 2 1 3.5 0 0 100 200 300 00 500 600 700 800 900 1000 3 2.5 2 1.5 1 0.5 0 0 100 200 300 00 500 600 700 800 900 1000 Exponential Singularvalue decay [see also Schwab et al.] 10 x 10 2.5 x 105 x 10 Figure : grid 8 6 0, (left) l 1 = 1, l 2 = 2, l 3 = 1 and (right) l 1 = 5, l 2 = 10, l 2 = 5. 1st row - Gaussian, 2-nd exponential and 3-rd spherical cov. func. 1 / 25
Sparse tensor decompositions of kernels cov(x, y) = cov(x y) We want to approximate C R N N, N = n d by C r = r k=1 V 1 k... V d k such that C C r ε. The storage of C is O(N 2 ) = O(n 2d ) and the storage of C r is O(rdn 2 ). To define V i k use SVD. tion Logo Approximate Lock-up all Vk i in the H-matrix format HKT format. See basic arithmetics in [Hackbusch, Khoromskij, Tyrtyshnikov]. 15 / 25
Tensor approximation Let M d ν=1 M (1) ν M (2) ν, C q ν=1 C (1) ν C (2) ν, φ where M ν (j), C ν (j) R n n, φ (j) ν R n, Example: for mass matrix M R N N holds r ν=1 φ (1) ν φ (2) ν, M = M (1) I + I M (1), where M (1) R n n is one-dimensional mass matrix. tion Logo Hypothesis: Lock-up the Kronecker rank of M stays small even for a more general domain with non-regular grid. 16 / 25
17 / 25 Suppose C = q ν=1 C(1) ν C ν (2) and φ = r Then tensor vector product is defined as Cφ = q r ν=1 j=1 The complexity is O(qrkn log n). j=1 φ(1) (C (1) ν φ (1) j ) (C (2) ν φ (2) j ). j φ (2) j.
Numerical examples of tensor approximations Gaussian kernel exp( h 2 ) has the Kroneker rank 1. The exponen. kernel exp( h) can be approximated by a tensor with low Kroneker rank r 1 2 3 5 6 10 C C r C 11.5 1.7 0. 0.1 0.035 0.007 2.8e 8 C C r 2 C 2 6.7 0.52 0.1 0.03 0.008 0.001 5.3e 9 18 / 25
Example Let G = [0, 1] 2, L h the stiffness matrix computed with the five-point formula. Then L h 2 8h 2 cos 2 (πh/2) < 8h 2. Lemma The (n 1) 2 eigenvectors of L h are u νµ (1 ν, µ n 1): u νµ (x, y) = sin(νπx) sin(µπy), (x, y) G h. The corresponding eigenvalues are λ νµ = h 2 (sin 2 (νπh/2) + sin 2 (µπh/2)), 1 ν, µ n 1. Use Lanczos method with the matrix in the HKT format to tion Logo compute Lock-up eigenpairs of L h v i = λ i v i, i = 1..N. Then we compare the computed eigenpairs with the analytically knowncenter eigenpairs. for Uncertainty 19 / 25
Application: covariance of the solution Let K be the stiffness matrix. For SPDE with stochastic RHS the eigenvalue problem and spectral decom. look like If we only want the covariance C f f l = λ l f l, C f = Φ f Λ f Φ T f. C u = (K K) 1 C f = (K 1 K 1 )C f = K 1 C f K T, one may with the KLE of C f = Φ f Λ f Φ T f reduce this to C u = K 1 C f K T = K 1 Φ f ΛΦ T f K T. 20 / 25
Higher order moments Let operator K be deterministic and Ku(θ) = α J Ku (α) H α (θ) = f(θ) = α J f (α) H α (θ), with u (α) = [u (α) 1,..., u(α) N ]T. Projecting onto each H α obtain Ku (α) = f (α). The KLE of f(θ) is f(θ) = f + l λl φ l (θ)f l = λl φ (α) l H α (θ)f l = α l H α (θ)f (α), α where f (α) = l λl φ (α) l f l. 21 / 25
The 3-rd moment of u is M (3) u = E u (α) u (β) u (γ) H α H β H γ = u (α) u (β) u (γ) c α α,β,γ α,β,γ c α,β,γ := E (H α (θ)h β (θ)h γ (θ)) = c (γ) α,β γ!, and c (γ) α,β := α!β!, g := (α + β + γ)/2. (g α)!(g β)!(g γ)! Using u (α) = K 1 f (α) = l λl φ (α) l K 1 f l and û l := K 1 f l, obtain M u (3) = t p,q,r û p û q û r, p,q,r t p,q,r := λ p λ q λ r α,β,γ φ (α) p φ (β) q φ (γ) where r c α,β,γ. 22 / 25
Conclusion Covariance matrices allow data sparse low-rank approximations. With application of H-matrices we extend the class of covariance functions to work with, allows non-regular discretisations of the cov. function on large spatial grids. Application of sparse tensor product allows computation of k-th moments. 23 / 25
Plans for Feature 1. Convergence of the Lanczos method with H-matrices 2. Implement sparse tensor vector product for the Lanczos method 3. HKT idea for d 3 dimensions 2 / 25
Thank you for your attention Thank you for your attention! Questions? 25 / 25