Sampling and Low-Rank Tensor Approximations

Transcription

1 Sampling and Low-Rank Tensor Approximations Hermann G. Matthies Alexander Litvinenko, Tarek A. El-Moshely +, Brunswick, Germany + MIT, Cambridge, MA, USA wire@tu-bs.de $Id: 2_Sydney-MCQMC.tex,v.3 22/2/2 6:52:28 hgm Exp $

2 Overview 2. Functionals of SPDE solutions 2. Computing the simulation 3. Parametric problems 4. Tensor products and other factorisations 5. Functional approximation 6. Emulation approximation 7. Examples and conclusion

3 Problem statement 3 We want to compute J k = E (Ψ k (, u e ( ))) = Ψ k (ω, u e (ω)) P(dω), where P is a probability measure on Ω, and u e is the solution of a PDE depending on the parameter ω Ω. A[ω](u e (ω)) = f(ω) a.s. in ω Ω, u e (ω) is a U-valued random variable (RV). To compute an approximation u M (ω) to u e (ω) via simulation is expensive, even for one value of ω, let alone for N J k Ψ k (ω n, u M (ω n )) w n n= Not all Ψ k of interest are known from the outset. Ω

4 Example: stochastic diffusion 4 Geometry 2 Aquifer D Model Simple stationary model of groundwater flow with stochastic data κ, f (κ(x, ω) u(x, ω)) = f(x, ω) x D R d & b.c. Solution is in tensor space S U =: W, e.g. W = L 2 (Ω, P) H (D) leads after Galerkin discretisation with U M = span{v m } M m= U to A[ω](u M (ω)) = f(ω) a.s. in ω Ω, where u M (ω) = M m= u m(ω)v m S U M..5

5 Realisation of κ(x, ω) 5

6 Solution example 6 Geometry 2.5 flow = flow out.5 2 Sources Dirichlet b.c Realization of κ Realization of solution Mean of solution Variance of solution x Pr{u(x) > 8}.5 y.5

7 Computing the simulation 7 To simulate u M one needs samples of the random field (RF) κ, which depends on infinitely many random variables (RVs). This has to be reduced / transformed Ξ : Ω [, ] s to a finite number s of RVs ξ = (ξ,..., ξ s ), with µ = Ξ P the push-forward measure: J k = Ψ k (ω, u e (ω)) P(dω) ˆΨk (ξ, u M (ξ)) µ(dξ). Ω [,] s This is a product measure for independent RVs (ξ,..., ξ s ). Approximate expensive simulation u M (ξ) by cheaper emulation. Both tasks are related by viewing u M : ξ u M (ξ), or κ : x κ(x, ) (RF indexed by x), or κ 2 : ω κ(, ω) (function valued RV), maps from a set of parameters into a vector space.

8 Parametric problems and RKHS 8 For each p in a parameter set P, let r(p) be an object in a Hilbert space V (for simplicity). With r : P V, denote U = span r(p) = span im r, then to each function r : P U corresponds a linear map R : U ˆR: R : U v r( ) v V ˆR = im R R P. (sometimes called a weak distribution) By construction R is injective. Use this to make ˆR a pre-hilbert space: φ, ψ ˆR : φ ψ R := R φ R ψ U. R is unitary on completion R which is a RKHS reproducing kernel Hilbert space with kernel ρ(p, p 2 ) = r(p ) r(p 2 ) U. Functions in R are in one-to-one correspondence with elements of U.

9 Covariance 9 If Q R P is Hilbert with inner product Q ; e.g. Q = L 2 (P, ν), define in U a positive self-adjoint map the covariance C = R R Cu v U = Ru Rv Q, has spectrum σ(c) R +, with spectral projectors E λ : C = λ de λ Similarly, define Ĉ : Q Q for φ, ψ Q such that Ĉ = RR by Ĉφ ψ Q = R φ R ψ U has same spectrum as C : σ(ĉ) = σ(c), and unitarily equivalent projectors Êλ = W E λ W : Ĉ = λ dêλ. Spectrum and projectors (σ(c), E λ ) are essence of r(p). Specifically, for φ, ψ L 2 (P, ν) we have R φ R ψ U = φ(p )ρ(p, p 2 )ψ(p 2 ) ν(dp ) ν(dp 2 ). P P

10 Covariance operator and SVD Cv = Spectral decomposition with projectors E λ λ de λ v = λ j e j v U e j + λ j σ p (C) R + \σ p (C) λ de λ v. C unitarily equivalent to multiplication operator M k with non-negative k: C = U M k U = (U M /2 k )(M /2 k U), with M /2 k = M k. This connects to the singular value decomposition (SVD) of R = V M /2 k U, with a (partial) isometry V. Often C has a pure point spectrum (e.g. C compact) last integral vanishes. In general to show tensors we have to invoke generalised eigenvectors and Gelfand triplets (rigged Hilbert spaces) for the continuous spectrum.

11 SVD, Karhunen-Loève-expansion, and tensors For sake of simplicity assume σ(c) = σ p (C). C = λ j e j U e j = λ j e j e j. j j (Rv)(p) = r(p) v U = j λj e j v U s j (p) with s j := Re j with R = j λj (s j e j ), or R = λj (e j s j ), r(p) = λj s j (p)e j, r S U. j j The singular value decomposition, a.k.a. Karhunen-Loève-expansion. A sum of rank- operators / tensors. In general C = R + λ e λ, e λ ϱ(dλ) with generalised eigenvectors e λ.

12 Examples and interpretations 2 If V is a space of centred random variables (RVs), r is a random field or stochastic process indexed by P, then Ĉ represented by the kernel ρ(p, p 2 ) is the covariance function. If in this case P = R d and moreover ρ(p, p 2 ) = c(p p 2 ) (stationary process / homogeneous field), then the diagonalisation U is effected by the Fourier transform, and the point spectrum is typically empty. If ν is a probability measure (ν(p) = ), and r is a V-valued RV, then C is the covariance operator. If P = {, 2,..., n} and R = R n, then ρ is the Gram matrix of the vectors r,..., r n. If n < dim V, the map R can be seen as a model reduction projector.

13 Factorisations / re-parametrisations 3 R serves as representation for Karhunen-Loève expansion. This is a factorisation of C. Some other possible ones: C = R R = (V M /2 k )(V M /2 k ) = C /2 C /2 = B B, where C = B B is an arbitrary one. Each factorisation leads to a representation all unitarily equivalent. (When C is a matrix, a favourite is Cholesky: C = LL ). Assume that C = B B and B : U H r U H. Select a orthonormal basis {e k } in H. Unitary Q : l 2 a = (a, a 2,...) k a ke k H. Approximation possible by injection Ps : R s l 2. Let r(a) := B Qa := R a (linear in a), i.e. R : l 2 U. Then R R = (B Q)(Q B) = B B = C.

14 Representations 4 Several representions for object r(p) U in a simpler space. The RKHS The Karhunen-Loève expansion based on spectral decomposition of C. The multiplicative spectral decomposition, as V M /2 k maps into U. Arbitrary factorisations C = B B. Analogous: consider Ĉ instead of C. If Q = L 2(P, ν) this leads to integral transforms, the kernel decompositions. These can all be used for model reduction, choosing a smaller subspace. Applied to RF κ(x, ω), and hence to u M (ω), yielding u M (ξ). Can again be applied to u M (ξ).

15 Functional approximation 5 Emulation replace expensive simulation u M (ξ) by inexpensive approximation / emulation u E (ξ) u M (ξ) ( alias response surfaces, proxy / surrogate models, etc.) Choose subspace S B S with basis {X β } B β=, make ansatz for each u m (ξ) β uβ mx β (ξ), giving u E (ξ) = m,β u β mx β (ξ)v m = m,β u β mx β (ξ) v m. Set U = (u β m) (M B). Sampling, we generate matrix / tensor U = [u M (ξ ),..., u M (ξ N )] = (u m (ξ n )) n m (M N).

16 Tensor product structure 6 Story does not end here as one may choose S = k S k, approximated by S B = K k= S B k, with S Bk S k. Solution represented( as a tensor of grade K + K ) in W B,N = k= S B k U N. For higher grade tensor product structure, more reduction is possible, but that is a story for another talk, here we stay with K =. With orthonormal X β one has N u β m = X β (ξ)u m (ξ) µ(dξ) w n X β (ξ n )u m (ξ n ). [,] s n= Let W = diag (w n ) (N N), X = (X β (ξ n )) (B N), hence U = U(W X T ). For B = N this is just a basis change.

17 Low-rank approximation 7 Focus on array of numbers U := [u m (ξ n )], view as matrix / tensor: N M U m,n e m M e n N, with unit vectors e n N R N, e m M R M. n= m= The sum has M N terms, the number of entries in U. Rank-R representation is approximation with R terms N M R U = U m,n e m M(e n N) T a l b T l = AB T, n= m= with A = [a,..., a R ] (M R) and B = [b,..., b R ] (N R). It contains only R (M + N) M N numbers. We will use updated, truncated SVD. This gives for coefficients U = U(W X T ) AB T (W X T ) = A(XW B) T =: A B T l=

18 Emulation instead of simulation 8 Let x(ξ) := [X (ξ),..., X B (ξ)] T. Emulator and low-rank emulator is u E (ξ) = Ux(ξ), and u L (ξ) := A B T x(ξ). Computing A, B: start with z samples U z = [u M (ξ ),..., u M (ξ z )]. Compute truncated, error controled SVD: M z U z M R W R R Σ ( z R V ) T ; then set A = W Σ /2, B = V Σ /2 B. For each n = z +,..., 2z, emulate u L (ξ n ) and evaluate residuum r n := r(ξ n ) := f(ξ n ) A[ξ n ](u L (ξ n )). If r n is small, accept u n A = u L(ξ n ), otherwise solve for u M (ξ n ) and set u n A = u M(ξ n ). Set U z2 = [u z+ A,..., u2z A ], compute updated SVD of [U z, U z2 ], A 2, B 2. Repeat for each batch of z samples.

19 Emulator in integration 9 J k = Ω To evaluate Ψ k (ω, u e (ω)) P(dω) ˆΨk (ξ, u M (ξ)) µ(dξ), [,] s we compute N J k w n ˆΨk (ξ n, u L (ξ n )). n= If we are lucky, we need much fewer than N samples to find the low-rank representation A, B for u L. This is cheap to compute from samples, and uses only little storage. In the integral the integrand is cheap to evaluate, and the low-rank representation can be re-used if a new (J k, Ψ k ) has to be evaluated.

20 Use in MC sampling solution sample 2 Example: Compressible RANS-flow around RAE air-foil. Sample solution turbulent kinetic energy pressure

21 Use in MC sampling solution storage 2 Inflow and air-foil shape uncertain. Data compression achieved by updated SVD: Made from 6 MC Simulations, SVD is updated every samples. M = 26, N = 6 Updated SVD: Relative errors, memory requirements: rank R pressure turb. kin. energy memory [MB].9e-2 4.e e-2 5.9e e-3.5e-4 4 Dense matrix R 26 6 costs 25 MB storage.

22 Use in QMC sampling mean 22 Trans-sonic flow with shock with N = 26 samples. Relative error for the density mean for rank R = 5,, 3, 5.

23 Use in QMC sampling variance 23 Trans-sonic flow with shock with N = 26 samples. Relative error for the density variance for rank R = 5,, 3, 5.

24 Conclusion 24 Random field discretisation and sampling can be seen as weak distribution with associated covariance. Analysis of associated linear map reveals essential structure. Factorisations of covariance lead to SVD (Karhunen-Loève expansion) and tensor products. Functional approximation to construct emulator. Sparse and inexpensive emulation.