Uncertainty Quantification and related areas - An overview

Transcription

1 Uncertainty Quantification and related areas - An overview Alexander Litvinenko, talk given at Department of Mathematical Sciences, Durham University Bayesian Computational Statistics & Modeling, KAUST Stochastic Numerics Group Alexander Litvinenko, talk given at Department of Mathematical Sciences, Uncertainty Durham Quantification University and related areas - An overview

2 The structure of the talk. Part I. Introduction to UQ. Part II. Low-rank tensors for representation of big/high-dimensional data 3. Part III. Inverse Problem via Bayesian Update 4. Part IV. R-INLA and advance numerics for spatio-temporal statistics 5. Part V. High Performance Computing, parallel algorithms

3 Part I. Introduction into UQ 3

4 Motivation to do Uncertainty Quantification (UQ) Motivation: there is an urgent need to quantify and reduce the uncertainty in multiscale-multiphysics applications. Nowadays computational predictions are used in critical engineering decisions. But, how reliable are these predictions? Example: Saudi Aramco currently has a simulator, TeraPOWERS, which runs trillion-cell simulation. How sensitive are these simulations w.r.t. unknown reservoir properties? My goal is development of UQ methods and low-rank algorithms relevant for applications. (pictures are taken from internet) 4

5 Uncertainty Quantification Consider { div(κ(x, ω) u(x, ω)) = f (x, ω) in G Ω, G R d, u = 0 on G, where κ(x, ω) - uncertain diffusion coefficient.. Efficient Analysis of High Dimensional Data in Tensor Formats, Espig, Hackbusch, Litvinenko., Matthies, Zander, 0.. Efficient low-rank approx. of the stoch. Galerkin matrix in tensor formats, Wähnert, Espig, Hackbusch, A.L., Matthies, PCE of random coefficients and the solution of stochastic PDEs in the Tensor Train format, Dolgov, Litvinenko, Khoromskij, Matthies, Application of H-matrices for computing the KL expansion, Khoromskij, Litvinenko, Matthies Computing 4 (-), 49-67, realizations of the solution u, the mean and quantiles Related work by R. Scheichl, Chr. Schwab, A. Teckentrup, F. Nobile, D. Kressner,... 5

6 Discretisation techniques Truncated Karhunen Loève Expansion (KLE): κ(x, ω) κ 0 (x) + L κ j g j (x)ξ j (θ(ω)), j= where θ = θ(ω) = (θ (ω), θ (ω),..., ), ξ j (θ) = κ j G (κ(x, ω) κ 0(x)) g j (x)dx.. S. Dolgov, B.N. Khoromskij, A. Litvinenko, H.G. Matthies, Computation of the Response Surface in the Tensor Train data format, arxiv preprint arxiv:406.6, 04. S. Dolgov, B.N. Khoromskij, A. Litvinenko, H.G. Matthies, Polynomial Chaos Expansion of Random Coefficients and the Solution of Stochastic Partial Differential Equations in the Tensor Train Format, IAM/ASA J. Uncertainty Quantification 3 (), 09-35, 05 6

7 Generalized Polynomial Chaos Expansions (gpce) Decompose ξ j (θ) from above: ξ j (θ) ξ (α) j α J M,p H α (θ), H α (θ) could be multivariate Hermite polynomials, α = (α,..., α M ). And, combining with KLE above, obtain L κ(x, ω) κ 0 (x) + κ j g j (x) j= ξ (α) j α J M,p H α (θ) Need a compact representation of a sum above. TENSORS! Can also apply gpce directly κ(x, ω) κ (α) (x)h α (θ). α J M,p 7

8 Part II. Low-rank tensor approximations of big data Goal: To provide fast and cheap numerical techniques for working with big and high-dimensional data. How?: By reducing linear algebra cost from O(n d ) to O(drn).

9 Curse of dimensionality Assume we have n d data. Our aim is to reduce storage/complexity from O(n d ) to O(dn). For n = 00 and d = 0, then just to store one needs = 0 TB. If we assume that a modern computer compares 0 7 numbers per second, then the total time for comparison 0 0 elements will be 0 3 seconds or years. In some chemical applications we had n = 00 and d = 00. how to compute the mean? how to compute maxima and minima? how to compute level sets, i.e. all elements from an interval [a, b]? how to compute the number of elements in an interval [a, b]? 9

10 Canonical (CP) and Tucker tensor formats Tensor of order d is a multidimensional array over a d-tuple index set I = I I d, A = [a i...i d : i l I l ] R I, I l = {,..., n l }, l =,.., d. Storage: O(n d ) O(dRn) and O(R d + drn). A. Litvinenko, D. Keyes, V. Khoromskaia, B.N. Khoromskij, H.G. Matthies, Tucker Tensor analysis of Matern functions in spatial statistics, preprint arxiv:7.0674, 07 0

11 Discretization of elliptic PDE Ku = f, where L M K := K l lµ, K l R N N, lµ R Rµ Rµ, l= µ= r M u := u j u jµ, u j R N, u jµ R Rµ, j= µ= R M f := f k g kµ, f k R N, g kµ R Rµ. k= µ= Efficient low-rank approximation of the stochastic Galerkin matrix in tensor formats, Wähnert, Espig, Hackbusch, Litvinenko, Matthies, 03. We analyzed tensor ranks (compression properties) of the stochastic Galerkin operator K.

12 Numerical Experiments D L-shape domain, N = 557 dofs. Total stochastic dimension is Mu = Mk + Mf = 0, there are JM,p = 3 PCE coefficients u= 3 X j= uj,0 0 O µ= ujµ R557 0 O R3. µ= Tensor u has entries 6 TB of memory. Instead we store only 3 ( ) entries.4 MB.

13 How to compute the variance in CP format Let u R r and ũ := u u d µ= r+ = n µ j= µ= d ũ jµ R r+, () then the variance var(u) of u can be computed as follows ũ, ũ r+ d r+ d var(u) = d µ= n = d µ µ= n ũ iµ, µ = r+ r+ d i= j= µ= n µ ũiµ, ũ jµ. i= µ= ( Numerical cost is O (r + ) d ) µ= n µ. j= ν= ũ jν

14 Level sets Now we compute level sets {ui : ui > b max u}, i i := (i,..., im+ ) for b {0., 0.4, 0.6, 0.}. The computing time for each b was 0 minutes. Key words: u is tensor, Newton methods, inverse u, sign(u), frequency, characteristic function. 3

15 Computation of level sets and frequency. To compute level sets and frequencies we need characteristic function.. To compute characteristic function we need sign(u) function. 3. To compute sign(u) function we need u. Proposition Let I R, u T, and χ I (u) its characteristic. We have L I (u) = χ I (u) u and rank(l I (u)) rank(χ I (u)) rank(u). The frequency F I (u) N of u respect to I is F I (u) = χ I (u),, where = d µ= µ, µ := (,..., ) T R nµ. 4

16 Part III. Inverse Problem via Bayesian Update 5

17 Developing of cheap Bayesian update surrogate. H.G. Matthies, E. Zander, B.V. Rosic, A. Litvinenko, Parameter estimation via conditional expectation: a Bayesian inversion, Advanced modeling and simulation in engineering sciences 3 (), 4, 06. Inverse Problems in a Bayesian Setting, H.G. Matthies, E. Zander, O. Pajonk, B. Rosic, A. Litvinenko. Computational Methods for Solids and Fluids Multiscale Analysis, ISSN: , 06 Related work by A. Stuart, Chr. Schwab, A. El Sheikh, Y. Marzouk, H. Najm, O. Ernst x f x a x y a y y f 0.9 z f z a z

18 Numerical computation of Bayesian Update surrogate Notation: ŷ measurements from engineers, y(ξ) forecast from the simulator, ε(ω) the Gaussian noise. Look for ϕ such that q(ξ) = ϕ(z(ξ)), z(ξ) = y(ξ) + ε(ω): ϕ ϕ = α J p ϕ α Φ α (z(ξ)) and minimize q(ξ) ϕ(z(ξ)) L, where Φ α are known polynomials (e.g. Hermite). Taking derivatives with respect to ϕ α : ϕ α q(ξ) ϕ(z(ξ)), q(ξ) ϕ(z(ξ)) = 0 α J p 6

19 Numerical computation of NLBU E q (ξ) qϕ β Φ β (z) + ϕ β ϕ γ Φ β (z)φ γ (z) ϕ α β J β,γ J = E qφ α (z) + ϕ β Φ β (z)φ α (z) β J = E [Φ β (z)φ α (z)] ϕ β E [qφ α (z)] = 0 α J. β J

20 Numerical computation of NLBU Now, rewriting the last sum in a matrix form, obtain the linear system of equations (=: A) to compute coefficients ϕ β : E [Φ α (z(ξ))φ β (z(ξ))] where α, β J, A is of size J J.. ϕ β. =. E [q(ξ)φ α (z(ξ))] A. Litvinenko, H.G. Matthies, Inverse problems and uncertainty quantification, arxiv preprint arxiv:3.504, 03.,

21 Numerical computation of NLBU Finally, the assimilated parameter q a will be z(ξ) = y(ξ) + ε(ω), ϕ = β J p ϕ β Φ β (z(ξ)) 6 q a = q f + ϕ(ŷ) ϕ(z), () κ f 4 κ a PDF κ 9

22 Example: D elliptic PDE with uncertain coeffs (κ(x, ξ) u(x, ξ)) = f (x, ξ), x [0, ] + Dirichlet random b.c. g(0, ξ) and g(, ξ). 3 measurements: u(0.3) =, s.d. 0., x(0.5) =, s.d. 0.3, x(0.) =, s.d κ(x, ξ): N = 00 dofs, M = 5, number of KLE terms 35, beta distribution for κ, Gaussian cov κ, cov. length 0., multi-variate Hermite polynomial of order p κ = ; RHS f (x, ξ): M f = 5, number of KLE terms 40, beta distribution for κ, exponential cov f, cov. length 0.03, multi-variate Hermite polynomial of order p f = ; b.c. g(x, ξ): M g =, number of KLE terms, normal distribution for g, Gaussian cov g, cov. length 0, multi-variate Hermite polynomial of order p g = ; p φ = 3 and p u = 3 0

23 Example: Updating of the parameter Figure: Prior and posterior (updated) parameter κ. Collaboration with Y. Marzouk, MIT, and TU Braunschweig. Together with H. Najm, Sandia Lab, we try to compare our technique with his advanced MCMC technique for chemical combustion eqn.

24 Example: updating of the solution u Figure: Original and updated solutions, mean value plus/minus,,3 standard deviations. Number of available measurements {0,,, 3, 5} [graphics are built in the stochastic Galerkin library sglib, written by E. Zander in TU Braunschweig]

25 Minimization of Uncert. in Num. Aerodynamics Involved: 7 German Universities, DLR, Airbus Duration: Quantification of airfoil geometry-induced aerodynamic uncertainties-comparison of approaches, Liu, Litvinenko, Schillings, Schulz, JUQ 07. Numerical Methods for Uncertainty Quantification and Bayesian Update in Aerodynamics Litvinenko, Matthies, chapter in Springer book, Vol, pp 6-5, 03. 3

26 Example: uncertainties in free stream turbulence v u v α α u v Random vectors v (θ) and v (θ) model free stream turbulence 4

27 Example: 3sigma intervals Figure: 3σ interval, σ standard deviation, in each point of RAE airfoil for the pressure (cp) and friction (cf) coefficients. 5

28 Mean and variance of density, tke, xv, zv, pressure mean density, variance of density, mean turbul. kinetic energy, x-velocity, z-velocity, a pressure (failed to compute) 6

29 Part IV. INLA and advance numerics for spatio-temporal statistics 7

30 Sparse matrices in INLA Integrated Nested Laplace Approximations (INLA) approximates Bayesian inference for latent Gaussian models (LGMs). Laplace method approximates high-dimensional integrals with a second order Taylor expansion and computes the integral analytically. A nested version of this idea, combined with sparse data techniques is INLA. My goal: improve sparse matrix techniques in R-INLA project

31 Two ways to find a low-rank tensor approx. of Matérn cov. Low-rank approximation of Matérn covariance 9

32 Part V-a. Parallel algorithms in environmental statistics 30

33 Improving of the moisture model Task: to improve statistical model, which predicts moisture Given: D-Grid with n.5mi locations Soil moisture latitude longitude High-resolution daily soil moisture data at the top layer of the Mississippi basin, U.S.A., Important for agriculture, defense,.... A. Litvinenko, HLIBCov: Parallel Hierarchical Matrix Approximation of Large Covariance Matrices and Likelihoods with Applications in Parameter Identification, preprint arxiv: , 07. A. Litvinenko, Y. Sun, M.G. Genton, D. Keyes, Likelihood Approximation With Hierarchical Matrices For Large Spatial Datasets, preprint arxiv: , 07 3

34 Improving of the moisture model Goal: To improve estimation of unknown statistical parameters in a spatial soil moisture field. Log-likelihood function with Matérn C(θ) and available data Z : L(θ) = n log(π) log C(θ) Z C(θ) Z. To identify: unknown parameters θ := (σ, ν, l). Collaboration with statisticians: M. Genton, Y. Sun, R. Huser, from KAUST. n = 5K, matrix setup 6 sec., compression rate 99.9% (0.4 GB against 006 GB). H-LU is done in 43 sec., error

35 Parameter identification l ν n, samples in thousands n, samples in thousands Synthetic data with known parameters (l, ν, σ ) = (0.5,, 0.5). Boxplots for l and ν for n =, 000 {64, 3,..., 4, }; 00 replicates 33

36 Part V-b. High Performance Computing and UQ 34

37 Parallel algorithms with distributed data Consider density driven groundwater flow with uncertain porosity and permeability (ρq) = 0, (3) t (φc) + (cq D c) = 0. (4) Mean and variance of porosity The mean (left) and the variances (right) of the porosity, [0, 600] [0, 50], φ(x) 0., var(φ(x)) (3. 0 5, ). We compute 00 scenarios in parallel, each scenario on 3 cores in parallel. Total: 00 3 = 5600 cores

38 Mean and variance of concentration The mean and variance of the concentration, computed via QMC. The number of time steps is 500; E(c) (0, ), var(c) (0, 0.06). Two different realizations of concentration: with 5 fingers (left) and 4 fingers (right). 36

39 Conclusion Introduced:. UQ and many examples,. Big data require a compact representation. We suggest low-rank tensor data techniques. 3. Tensor techniques requires to redefine standard statistical algorithms 4. Bayesian update surrogate ϕ (as a linear, quadratic,... approximation) 5. Applications in aerodynamics, geostatistics, reservoirs, environmental statistics 6. HPC applications: parallel algorithms for UQ on a supercomputer 37

40 Literature. A. Litvinenko, Application of Hierarchical matrices for solving multiscale problems, PhD Thesis, Leipzig University, Germany, B.N. Khoromskij, A. Litvinenko, Domain decomposition based H-matrix preconditioners for the skin problem, Domain Decomposition Methods in Science and Engineering XVII, pp 75-, A. Litvinenko, Documentation for the Hierarchical Domain Decomposition (HDD) method, Technical report, Vol 5, pp -33, Max-Planck Institute for applied mathematics in the science in Leipzig, A. Litvinenko, Partial inversion of elliptic operator to speed up computation of likelihood in Bayesian inference, arxiv preprint , W. Nowak, A. Litvinenko, Kriging and spatial design accelerated by orders of magnitude: Combining low-rank covariance approximations with FFT-techniques, Mathematical Geosciences 45 (4), 4-435, 03 3