Sampling and low-rank tensor approximation of the response surface tifica Alexander Litvinenko 1,2 (joint work with Hermann G. Matthies 3 ) 1 Group of Raul Tempone, SRI UQ, and 2 Group of David Keyes, Extreme Computing Research Center KAUST, 3 TU Braunschweig, Germany http://sri-uq.kaust.edu.sa/
KAUST Figure : KAUST campus, 5 years old, approx. 7000 people (include 1400 kids), 100 nations. 2 / 43
Stochastic Numerics Group at KAUST 3 / 43
4 / 43
Points of my study and of my work. 5 / 43
The structure of the talk 1. Results of MUNA (input: α, Ma, output CL, CD, cp, cf) 2. Low-rank response surface 3. Numerics (input: α, Ma, output CL, CD, density, pressure,...) 4. Use the response surface to get a good start value 5. Random geometry of the airfoil 6. Comparison of 5 methods 6 / 43
Results of MUNA project (2007-2011) are published 390 pages, our results on pp. 265-281 7 / 43
Sparse Gauss-Hermite grid, 13 points. Assume that RVs α and Ma are Gaussian with mean st. dev. σ/mean σ α 2.79 0.1 0.036 Ma 0.734 0.005 0.007 Then uncertainties in the solution lift CL and drag CD are CL 0.853 0.0174 0.02 CD 0.0206 0.003 0.146 8 / 43
Example 1: 500 MC realisations of cp in dependence on α i and Ma i 9 / 43
Example 2: 500 MC realisations of cf in dependence on α i and Ma i 10 / 43
Example 3: 5% and 95% quantiles for cp from 500 MC realisations. 11 / 43
Example 4: 5% and 95% quantiles for cf from 500 MC realisations. 12 / 43
Compute difference between the mean solution and the deterministic solution (α = 2.74, Ma = 0.73). The difference is large! It is comparable with the lowest physical value! (a) deterministic turbulent viscosity (2e 5, 3.6e 4) (b) deterministic density (0.45, 1.25) 13 / 43
Response surface Let RF q(x, θ), θ = (θ 1,..., θ M,...) is approximated: q(x, θ) = β J q β (x)h β (θ) (1) q β (x) = 1 H β (θ)q(x, θ) P(dθ) 1 n q H β (θ i )q(x, θ i )w i, β! Θ β! where n q - number of quadrature points. Using low-rank format, obtain tion Logo q β (x) Lock-up = 1 β! [q(x, θ 1),..., q(x, θ nq )] [H β (θ 1 )w 1,..., H β (θ nq )w nq ] T i=1 14 / 43
PCE coefficients Denote c β := 1 β! [H β(θ 1 )w 1,..., H β (θ nq )w nq ] T R nq and approximate the set of realisations in low-rank format: [q(x, θ 1 ),..., q(x, θ nq )] AB T. The matrix of all PCE coefficients will be R N J [...q β (x)...] AB T [...c β...], β J. (2) Put all together, obtain low-rank representation of RS q(x, θ) = q β (x)h β (θ) = [...q β (x)...]h T (θ), (3) β J where H(θ) = (..., H β (θ),...). 15 / 43
Application of low-rank response surface Now, having discretised RS q(x, θ) q(θ) = AB T [...c β...]h T (θ) (4) Sample RV θ 10 6 times and then use the obtained sample to compute errorbars, quantiles, cumulative density function. 16 / 43
Use in MC sampling solution storage Inflow and air-foil shape uncertain. Data compression achieved by updated SVD: Made from 600 MC Simulations, SVD is updated every 10 samples. n = 260, 000 Z = 600 Updated SVD: Relative errors, memory requirements: rank k pressure turb. kin. energy memory [MB] 10 1.9e-2 4.0e-3 21 20 1.4e-2 5.9e-3 42 50 5.3e-3 1.5e-4 104 Dense matrix R 260000 600 costs 1250 MB storage. 17 / 43
Conjunction of two low-rank matrices W k = [q(x, θ 1 ),..., q(x, θ Z )], W = [q(x, θ Z+1 ),..., q(x, θ Z+m )] Suppose W k = AB T R n Z is given Suppose W R n m contains new m solution vectors Compute C R n k and D R m k such that W CD T. Build Anew := [A C] R n 2k and B T new = blockdiag[b T D T ] R 2k (Z+m) Rank-k truncation of Wnew = AnewB T new costs O((n + Z + m)k 2 + k 3 ) 18 / 43
Mean and variance in the rank-k format u := 1 Z Cost is O(k(Z + n)). Z u i = 1 Z i=1 Z A b i = Ab. (5) i=1 C = 1 Z 1 W cw T c 1 Z 1 U kσ k Σ T k UT k. (6) Cost is O(k 2 (Z + n)). Lemma: Let W W k 2 ε, and u k be a rank-k approximation of the mean u. Then a) u u k ε Z, b) C C k 1 Z 1 ε2. 19 / 43
Tensor product structure Story does not end here as one may choose S = k S k, approximated by S B = K k=1 S B k, with S Bk S k. Solution represented( as a tensor of grade K + 1 K ) in W B,N = k=1 S B k U N. For higher grade tensor product structure, more reduction is possible, but that is a story for another talk, here we stay with K = 1. 20 / 43
Numerics Numerical experiments Compare MC with PCE-137, PCE-281 RAE-2822, Case 1, α = 1.93 and Ma = 0.676: (mean and variance) No difference, 137 is enough. Compare MC with PCE-29, PCE-201 tion Logo RAE-2822, Lock-up Case 9, α = 2.79 and Ma = 0.73: (mean and variance) 21 / 43
Case 1, density (a) ρ PCE137 ρ MC F = 0.5534 relative error: 0.00222 (b) ρ PCE281 ρ MC F = 0.5670 relative error: 0.00228 (c) var(ρ PCE137 ) var(ρ MC ) F = 0.25 (d) var(ρ PCE281 ) var(ρ MC ) F = 0.25 22 / 43
Case 1, Pressure (a) p PCE137 p MC F = 0.7114 relative error: 0.00282 (b) p PCE281 p MC F = 0.7240 relative error: 0.00287 (c) var(p PCE137 ) var(p MC ) F = 0.47 (d) var(p PCE281 ) var(p MC ) F = 0.47 23 / 43
Case 9, density (a) ρ PCE29 ρ MC F = 7.0 relative error: 0.0143 (b) ρ PCE201 ρ MC F = 7.2 relative error: 0.0148 (c) var(ρ PCE29 ) var(ρ MC ) F = 0.789 (d) var(ρ PCE201 ) var(ρ MC ) F = 0.621 24 / 43
Case 9, Pressure (a) p PCE29 p MC F = 9.0311 relative error: 0.01820 (b) p PCE201 p MC F = 9.3225 relative error: 0.01878 (c) var(p PCE29 ) var(p MC ) F = 1.3 (d) var(p PCE201 ) var(p MC ) F = 1.03 25 / 43
Relative error, density mean, trans-sonic flow, Z = 2600 samples 26 / 43
Relative error, density variance, trans-sonic flow, Z = 2600 27 / 43
Figure : Comparison of the mean pressures computed with PCE and with MC. (Left) p := p PCE137 p MC, Case 1 without shock, (Right) p := p PCE201 p MC, Case 9 with shock. 28 / 43
Sampling of response surface with residual Collocation points are θ l, l = 1..Z. Algorithm: 1. Compute RS q(x, θ) AB T [...c β...]h T (θ) from l points. 2. (l = l + 1), evaluate RS in θ l+1, obtain q(x, θ l+1 ). 3. Compute residual r(q(x, θ)). Only if r is large, solve expensive determ. problem. 4. Update A, B T, [...c β...] and go to (2). If we are lucky, we solve the determ. problem only few times, otherwise we must solve the determ. problem Z times for all tion Logo θ 1,...,θ Lock-up Z. 29 / 43
Errors, Case 1 Figure : (left) Relative errors in the Frobenius and the maximum norms for pressure and density. (right) 10 points (α, Ma) were chosen in the neigbourhood of α = 1.93 and Ma = 0.676. 30 / 43
Density, mean and variance The mean density and variance of the density. Case 9, RAE-2822. 31 / 43
Density with preconditioning failed Density evaluated from two different PCE-based response surfaces (of order p = 2 and p = 4). Failed. 32 / 43
Uncertain geometry of the airfoil Uncertain geometry of the airfoil. Comparison of 5 methods. 33 / 43
Uncertain geometry of the airfoil [1]. of airfoil geometry-induced aerodynamic uncertainties - comparison of approaches, D. Liu, A. Litvinenko,C. Schillings, V. Schulz, AIAA, 2015 Input: ψ (x, ω) - Gaussian RF, R(x, ω) = s(x) arccos (1 2Φ (ψ (x, ω))), G(ω) = G + R(x, ω) n(x), perturbed airfoil 34 / 43
Uncertainty quantification - comparison of methods To compare the efficiency of methods in estimating the statistics. Methods: gradient-employing gradient-assisted point-collocation polynomial chaos (GAPC) gradient-assisted radial basis function (GARBF) gradient-enhanced Kriging (GEK) not gradient-employing quasi-monte Carlo quadrature (QMC), with low discrepancy sequence polynomial chaos (PC), with coefficients estimated by sparse grid quadrature Criteria: computation cost for a certain accuracy in the statistics. Cost: measured in elapse time-penalized sample number M M = 2N for gradient-employing method M = N for others Accuracy: judged by comparing with a reference statistics obtained by a QMC of N = 5 10 5. 35 / 43
Three realizations of perturbed geometry 0.08 0.06 0.04 Original 1st realization 2nd realization 3rd realization 0.02 y 0 0.02 0.04 0.06 0.08 0 0.2 0.4 0.6 0.8 1 x Target statistics: means µ L and µ D, standard deviations σ L and σ D, exceedance probabilities P L,κ = Pro{C L µ L κ σ L } and P D,κ = Pro{C D µ D + κ σ D } with κ = 2, 3. 36 / 43
On estimating the mean of C L On estimating the stdv of C L 1e 03 1e 04 QMC PC SGH GEK GEPC GERBF 1e 03 1e 04 QMC PC SGH GEK GEPC GERBF Error Error 1e 05 1e 05 1e 06 10 1 10 2 10 3 M 10 1 10 2 10 3 M 1e 01 On estimating P L,2 On estimating P L,3 1e 02 1e 02 Error QMC 1e 03 PC SGH GEK GEPC GERBF tion Logo 1e 04 Lock-up 3ς 1 Error 1e 03 1e 04 QMC PC SGH GEK GEPC GERBF 3ς 1 10 1 10 2 10 3 M 10 1 10 2 10 3 M Figure : Absolute errors 37 / 43
On estimating the mean of C D On estimating the stdv of C D 1e 04 QMC PC SGH GEK GEPC GERBF 1e 04 QMC PC SGH GEK GEPC GERBF Error 1e 05 Error 1e 05 1e 06 10 1 10 2 10 3 M 1e 06 10 1 10 2 10 3 M On estimating P D,2 On estimating P D,3 1e 02 1e 01 1e 02 QMC PC SGH GEK GEPC GERBF 3ς 1 Error 1e 03 QMC PC SGH GEK GEPC 1e 04 GERBF 3ς 1 Error 1e 03 1e 04 10 1 10 2 10 3 M 10 1 10 2 10 3 M Figure : Absolute errors 38 / 43
Figure : Estimated pdf (in dash line) of C L by QMC, GEK, GEPC and GERBF at M = 33. 39 / 43 pdf of C L by QMC, M=33 pdf of C L by GEK, M=33 100 80 reference pdf pdf by QMC 100 80 reference pdf pdf by GEK 60 60 pdf pdf 40 40 20 20 0 0.79 0.8 0.81 0.82 0.83 C L 0 0.79 0.8 0.81 0.82 0.83 C L pdf of C L by GEPC, M=33 pdf of C L by GERBF, M=33 100 80 reference pdf pdf by GEPC 100 80 reference pdf pdf by GERBF 60 60 pdf pdf 40 40 20 0 0.79 0.8 0.81 0.82 0.83 C L 20 0 0.79 0.8 0.81 0.82 0.83 C L
Figure : Estimated pdf (in dash line) of C D by QMC, GEK, GEPC and GERBF at M = 33. 40 / 43 1000 800 pdf of C D by QMC, M=33 reference pdf pdf by QMC 700 600 500 pdf of C D by GEK, M=33 reference pdf pdf by GEK 600 400 pdf 400 pdf 300 200 200 100 700 600 500 0 3 4 5 6 7 8 C L x 10 3 pdf of C D by GEPC, M=33 reference pdf pdf by GEPC 0 3 4 5 6 7 8 C L x 10 3 700 600 500 pdf of C D by GERBF, M=33 reference pdf pdf by GERBF pdf 400 300 200 pdf 400 300 200 100 0 3 4 5 6 7 8 C L x 10 3 100 0 3 4 5 6 7 8 C L x 10 3
Uncertainty quantification - conclusion Conclusion: Gradient-employing surrogate methods outperform the others GEK seems more efficient than other gradient-employing methods, especially when M is small, and when estimating the far-end exceedance probability and the pdf. Since PC only has two data point, its performance in this comparison may not indicate its asymptotic capacity. 41 / 43
Acknowledgement 1. Nathalie Rauschmayr for pictures 2. Project MUNA, German Luftfahrtforschungsprogramm funded by the Ministry of Economics (BMWA). 3. Elmar Zander for sglib. Type in your terminal git clone git://github.com/ezander/sglib.git initialize all variables, run startup.m You will find: generalised PCE, sparse grids, (Q)MC, stochastic Galerkin, linear solvers, KLE, covariance matrices, statistics, quadratures (multivariate Chebyshev, Laguerre, Lagrange, Hermite ) etc There are: many examples, many test, rich demos To 42 / 43
Literature 1. PCE of random coefficients and the solution of stochastic partial differential equations in the Tensor Train format, S. Dolgov, B. N. Khoromskij, A. Litvinenko, H. G. Matthies, 2015/3/11, arxiv:1503.03210 2. Efficient analysis of high dimensional data in tensor formats, M. Espig, W. Hackbusch, A. Litvinenko, H.G. Matthies, E. Zander Sparse Grids and Applications, 31-56, 40, 2013 3. Application of hierarchical matrices for computing the Karhunen-Loeve expansion, B.N. Khoromskij, A. Litvinenko, H.G. Matthies, Computing 84 (1-2), 49-67, 31, 2009 4. Efficient low-rank approximation of the stochastic Galerkin matrix in tensor formats, M. Espig, W. Hackbusch, A. Litvinenko, H.G. Matthies, P. Waehnert, Comp. & Math. with Appl. 67 (4), 818-829, 2012 5. Numerical Methods for Uncertainty and Bayesian Update in Aerodynamics, A. Litvinenko, H. G. Matthies, Book Management and Minimisation of Uncertainties and Errors in Numerical Aerodynamics pp 265-282, 2013 43 / 43