Towards Large-Scale Computational Science and Engineering with Quantifiable Uncertainty

Size: px

Start display at page:

Download "Towards Large-Scale Computational Science and Engineering with Quantifiable Uncertainty"

Piers Perry
5 years ago
Views:

1 Towards Large-Scale Computational Science and Engineering with Quantifiable Uncertainty Tan Bui-Thanh Computational Engineering and Optimization (CEO) Group Department of Aerospace Engineering and Engineering Mechanics Institute for Computational Engineering and Sciences (ICES) The University of Texas at Austin Joint work with Carsten Burstedde, Omar Ghattas, James R. Martin, Georg Stadler, and Lucas Wilcox EU Regional School, Sep 23, 215 Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 1 / 44

2 (Very) Small-Scale Inverse problems u 1 w 2 + u 2 w + u 3 = Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 2 / 44

3 (Very) Small-Scale Inverse problems u 1 w 2 + u 2 w + u 3 = Forward problem Given: u 1, u 2, u 3 Desire: Compute solution w = u 2 ± u 2 2 4u 1u 3 2u 1 Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 2 / 44

4 (Very) Small-Scale Inverse problems u 1 w 2 + u 2 w + u 3 = Forward problem Given: u 1, u 2, u 3 Desire: Compute solution w = u 2 ± u 2 2 4u 1u 3 2u 1 Inverse problem Given: w Desire: Compute u 1, u 2, u 3 = Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 2 / 44

5 (Very) Small-Scale Inverse problems u 1 w 2 + u 2 w + u 3 = Forward problem Given: u 1, u 2, u 3 Desire: Compute solution w = u 2 ± u 2 2 4u 1u 3 2u 1 Inverse problem Given: w Desire: Compute u 1, u 2, u 3 =???? Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 2 / 44

6 Large-Scale Inverse problems Observations y obs = Computer Prediction w (u) + noise η (e u w) = in Ω e u w n = Bi w on Ω \ Γ R, e u w n = 1 on Γ R, Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 3 / 44

7 Large-Scale Inverse problems Observations y obs = Computer Prediction w (u) + noise η (e u w) = in Ω e u w n = Bi w on Ω \ Γ R, e u w n = 1 on Γ R, Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 3 / 44

8 Large-Scale Inverse problems y obs w (u) 2 L Observations y obs = Computer Prediction w (u) + noise η (e u w) = in Ω e u w n = Bi w on Ω \ Γ R, e u w n = 1 on Γ R, Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 3 / 44

9 Large-Scale Inverse problems subject to min w 1 2 y obs w (u) 2 L (e u w) = in Ω e u w n = Bi w on Ω \ Γ R, e u w n = 1 on Γ R, Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 3 / 44

$(u) 2 L + 1 2 u u 2 C (e u w) = in Ω e u w n = Bi w on Ω \$ Γ R, e u w n = 1 on Γ R, Tan Bui-Thanh (UT Austin)

10 Large-Scale Inverse problems subject to min w 1 2 y obs w (u) 2 L u u 2 C (e u w) = in Ω e u w n = Bi w on Ω \ Γ R, e u w n = 1 on Γ R, Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 3 / 44

11 Imperfect path to knowledge Challenges due to noise/errors! Courtesy of Oden et al. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 4 / 44

12 Tentative Plan for the Lectures Contents 1 Basic Bayesian inverse framework (lecture-oriented) Probability space, random variables, distribution Conditional probability, Bayes formula, prior, likelihood, posterior Construction of likelihood and prior Relation between Bayesian inverse and deterministic inverse problems Monte Carlo and classical limit theorems Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 5 / 44

13 Tentative Plan for the Lectures Contents 1 Basic Bayesian inverse framework (lecture-oriented) Probability space, random variables, distribution Conditional probability, Bayes formula, prior, likelihood, posterior Construction of likelihood and prior Relation between Bayesian inverse and deterministic inverse problems Monte Carlo and classical limit theorems 2 Advanced topics (lecture-oriented) Advanced MCMC techiniques (Langevin, stochastic Newton, Hamiltonian, randomized MAP) Bayesian inversion in infinite dimensions Discretization-invariant MCMC methods Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 5 / 44

14 Tentative Plan for the Lectures Contents 1 Basic Bayesian inverse framework (lecture-oriented) Probability space, random variables, distribution Conditional probability, Bayes formula, prior, likelihood, posterior Construction of likelihood and prior Relation between Bayesian inverse and deterministic inverse problems Monte Carlo and classical limit theorems 2 Advanced topics (lecture-oriented) Advanced MCMC techiniques (Langevin, stochastic Newton, Hamiltonian, randomized MAP) Bayesian inversion in infinite dimensions Discretization-invariant MCMC methods 3 Large-scale Bayesian inverse problems (seminar-oriented) large-scale statistical inverse problems Big-data in large-scale inverse problems Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 5 / 44

Large-scale computation under uncertainty Inverse electromagnetic scattering Randomness Random errors in measurements are unavoidable Inadequacy of the mathematical model (Maxwell equations)

15 Large-scale computation under uncertainty Inverse electromagnetic scattering Randomness Random errors in measurements are unavoidable Inadequacy of the mathematical model (Maxwell equations) Challenge How to invert for the invisible shape/medium using computational electromagnetics with O ( 1 6) degree of freedoms? Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 6 / 44

Challenge How to image the earth interior using forward computational model with with O ( 1 9)

16 Large-scale computation under uncertainty Full wave form seismic inversion Randomness Random errors in seismometer measurements are unavoidable Inadequacy of the mathematical model (elastodynamics) Challenge How to image the earth interior using forward computational model with with O ( 1 9) degree of freedoms? Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 7 / 44

17 Outline 1 Statistical inverse problem in electromagnetics (Reduced-then-Sample) 2 Ultra-scale Seismic Wave Inversion (Sample-then-Reduce) 3 Big-Data in Inverse problems (if time permits) 4 Summary Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 8 / 44

18 Outline 1 Statistical inverse problem in electromagnetics (Reduced-then-Sample) 2 Ultra-scale Seismic Wave Inversion (Sample-then-Reduce) 3 Big-Data in Inverse problems (if time permits) 4 Summary Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 9 / 44

19 Inverse Shape Electromagnetic Scattering Problem Maxwell Equations: E = µ H t, H = ɛ E t, (Faraday) (Ampere) E: Electric field, H: Magnetic field, µ: permeability, ɛ: permittivity Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 1 / 44

20 Inverse Shape Electromagnetic Scattering Problem Maxwell Equations: E = µ H t, H = ɛ E t, (Faraday) (Ampere) E: Electric field, H: Magnetic field, µ: permeability, ɛ: permittivity Forward problem (discontinuous Galerkin discretization) y = G(u) where G maps shape parameters u to electric/magnetic field y at the measurement points Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 1 / 44

21 Inverse Shape Electromagnetic Scattering Problem Maxwell Equations: E = µ H t, H = ɛ E t, (Faraday) (Ampere) E: Electric field, H: Magnetic field, µ: permeability, ɛ: permittivity Forward problem (discontinuous Galerkin discretization) y = G(u) where G maps shape parameters u to electric/magnetic field y at the measurement points Inverse Problem Given (possibly noise-corrupted) measurements on y, infer u? Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 1 / 44

22 The Bayesian Statistical Inversion Framework Bayes Theorem Solution to the inverse problem is given as a posterior PDF over parameter space: π post (u y obs ) π pr (u)π like (y obs u) Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 11 / 44

23 The Bayesian Statistical Inversion Framework Bayes Theorem Solution to the inverse problem is given as a posterior PDF over parameter space: π post (u y obs ) π pr (u)π like (y obs u) Prior knowledge: The obstacle is smooth: ( 2π ) π pr (u) exp λ r (u)dθ Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 11 / 44

24 The Bayesian Statistical Inversion Framework Bayes Theorem Solution to the inverse problem is given as a posterior PDF over parameter space: π post (u y obs ) π pr (u)π like (y obs u) Prior knowledge: The obstacle is smooth: ( 2π ) π pr (u) exp λ r (u)dθ Likelihood: Additive Gaussian noise, for example, π like (y obs u) exp ( 12 ) G(u) y obs 2Σnoise Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 11 / 44

25 Challenge: Expensive Forward Solve 2π π post (u y obs ) exp λ r (u)dθ 1 2 G(u) y obs 2 Σ noise }{{} Expensive forward model:y=g(u) Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 12 / 44

26 Challenge: Expensive Forward Solve 2π π post (u y obs ) exp λ Approximate the likelihood Reduced basis method, polynomial chaos, and etc r (u)dθ 1 2 G(u) y obs 2 Σ noise }{{} Expensive forward model:y=g(u) Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 12 / 44

27 Challenge: Expensive Forward Solve 2π π post (u y obs ) exp λ Approximate the likelihood Reduced basis method, polynomial chaos, and etc Approximate the posterior r (u)dθ 1 2 G(u) y obs 2 Σ noise }{{} Expensive forward model:y=g(u) 1 The posterior is nicer than the likelihood due to the prior contribution 2 The posterior is scalar function of parameters u only We propose a Hessian-based Adaptive Gaussian Process response surface Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 12 / 44

28 Hessian-based Adaptive Gaussian Process Main idea (mitigating the curse of dimensionality) 1 Use Adaptive Sampling Algorithm to find the modes 2 Approximate the covariance matrix (Hessian inverse) 3 Partition parameter space using membership probabilities 4 Approximate the posterior with local Gaussian in subdomains 5 Glue all the local Gaussian approximations Exact density MemProb 1 MemProb 2 MemProb Exact density GP predictor d(m) m Membership probabilities m GP predictor versus exact Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 13 / 44

29 Details in: Bui-Thanh, T., Ghattas, O., and Higdon, D., Adaptive Hessian-based Non-stationary Gaussian Process Response Surface Method for Probability Density Approximation with Application to Bayesian Solution of Large-scale Inverse Problems, SIAM Journal on Scientific Computing, 34(6), pp. A2837 A2871, 212. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 14 / 44 Inverse shape electromagnetic scattering discontinuous Galerkin discretization with 8,892 state variables 24 shape parameters 1 million MCMC simulations for the Gaussian process response surface % credibility envelope Sample posterior mean Exact shape.3 Deterministic solution

30 Details in: Bui-Thanh, T., Ghattas, O., and Higdon, D., Adaptive Hessian-based Non-stationary Gaussian Process Response Surface Method for Probability Density Approximation with Application to Bayesian Solution of Large-scale Inverse Problems, SIAM Journal on Scientific Computing, 34(6), pp. A2837 A2871, 212. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 14 / 44 Inverse shape electromagnetic scattering discontinuous Galerkin discretization with 8,892 state variables 24 shape parameters 1 million MCMC simulations for the Gaussian process response surface % credibility envelope Sample posterior mean Exact shape.3 Deterministic solution.2.1 Gaussian process Exact Posterior Offline time 33 hours hours Gaussian process Exact Posterior Online time.96 hours hours

31 Outline 1 Statistical inverse problem in electromagnetics (Reduced-then-Sample) 2 Ultra-scale Seismic Wave Inversion (Sample-then-Reduce) 3 Big-Data in Inverse problems (if time permits) 4 Summary Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 15 / 44

32 Full wave form seismic wave inversion E t = 1 ( v + T v ), 2 ρ v t = (CE) + f Strain-velocity formulation I: fourth-order identity tensor, I: second-order identity tensor, f: external volumetric forces, C: four-order material tensor. Inverse problem statement Animated by Greg Abram E: strain tensor, v: velocity vector, ρ: density, e i : ith unit vector, Earth surface velocity at given locations is recorded Infer the wave velocities c s = µ/ρ and c p = (λ + 2µ) /ρ Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 16 / 44

33 Infinite dimensional Bayesian statistical inference Approach 1: Linearized Bayesian solution about the MAP Compute u MAP = arg min 1 y obs w (u) 2 L u u 2 C Linearize the forward map about the MAP y obs = f + A (u) + η Posterior becomes a Gaussian measure posterior mean u y obs µ = N (m, C), m = E [u] = u + C A (Γ + AC A ) 1 ( y obs f Au ) posterior covariance operator C = ( A Γ 1 A + C 1 ) 1 Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 17 / 44

34 Infinite dimensional Bayesian statistical inference Approach 1: Linearized Bayesian solution: Low rank approximation posterior covariance operator: A low rank approximation C = ( A Γ 1 A + C 1 = C 1/2 ) 1 ( C 1/2 A Γ 1 A C 1/2 + I C 1/2 (V r Λ r V r + I) 1 C 1/2 = C C 1/2 V r D r V rc 1/2 ) 1C 1/2 Low rank approximation only involves incremental forward and incremental adjoint solve Then use Sherman-Morrison-Woodbury Relative to the prior uncertainty, the posterior uncertainty is reduced when observations are made. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 18 / 44

$Approach 1: Compactness of the Hessian in inverse acoustic scattering Theorem Let (1 n) C m,α, where n is the refractive index, m N {}, α (, 1). The Hessian is a compact operator everywhere.$

35 Linearized Bayesian solution: Why low rank approximation? Approach 1: Compactness of the Hessian in inverse acoustic scattering Theorem Let (1 n) C m,α, where n is the refractive index, m N {}, α (, 1). The Hessian is a compact operator everywhere. Coupled FEM-BEM method Eigenvalues of Gauss-Newton Hessian Details in: T. Bui-Thanh and O. Ghattas, Analysis of the Hessian for inverse scattering problems. Part II: Inverse medium scattering of acoustic waves. Inverse Problems, 28, 552, 212. T. Bui-Thanh and O. Ghattas, Analysis of the Hessian for inverse scattering problems. Part I: Inverse shape scattering of acoustic waves. Inverse Problems 212 Highlights Collection, 28, 551, 212. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 19 / 44

36 Infinite dimensional Bayesian statistical inference Approach 1: Linearized Bayesian solution: Low rank approximation posterior covariance operator: A low rank approximation C = ( A Γ 1 A + C 1 = C 1/2 ) 1 ( C 1/2 A Γ 1 A C 1/2 + I C 1/2 (V r Λ r V r + I) 1 C 1/2 = C C 1/2 V r D r V rc 1/2 ) 1C 1/2 Low rank approximation only involves incremental forward and incremental adjoint solve Then use Sherman-Morrison-Woodbury Relative to the prior uncertainty, the posterior uncertainty is reduced when observations are made. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 2 / 44

37 hp non-conforming discontinuous Galerkin method Elastic-Acoustic coupling as conservation laws P mi e P e mi + M 6 M 7 M 4 M 5 M 3 M 1 M 2 P mi ei P ei mi 3 4 seven mortars 1 2 nonconforming hexahedral elements using Kopriva s mortar approach for hyperbolic equations theory for general hp-non-conforming dg implementation for h-non-conforming dg with 2:1 balance tensor product Lagrange basis on the Legendre-Gauss-Lobatto (LGL) nodes LGL quadrature (diagonal mass matrix) time integration by classical 4-stage/RK4 integrated parallel mesh generation/adaptivity () July 15, / 1 Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 21 / 44

38 Convergence for non-conforming hp-discretization Theorem Assume q e [H se (D e )] d, s e 3/2 with d = 6 for electromagnetic case and d = 12 for elastic acoustic case. In addition, suppose q d () = Πq(), and the mesh is affine and non-conforming. Then, the discontinuous Galerkin spectral element solution q d converges to the exact solution q, i.e., there exists a constant C that depends only on the angle condition of D e, s, and the material constants µ and ε (λ and µ for elastic acoustic case) such that q (t) q d (t) D N el,d C e h σe e q (t) Ne se [H se (D e ) d ] + C e t hσe 1/2 e max N q (t) e se 1/2 [H [,t] se (De )], d with h e = diam (D e ), σ e = min {p e + 1, s e }, and Hs (D e ) denoting the usual Sobolev norm Details in: T. Bui-Thanh and O. Ghattas, Analysis of an hp-non-conforming discontinuous Galerkin spectral element method for wave propagations, SIAM Journal on Numerical Analysis, 5(3), pp , 212. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 22 / 44

39 Scalability of global seismic wave propagation on Jaguar Strong scaling: 3rd order DG, 16,195,864 elements, 9.3 billion DOFs #cores time [ms] elem/core efficiency [%] Strong scaling: 6th order DG, 17 million elements, 525 billion DOFs # cores meshing wave prop par eff Tflops time (s) per step (s) wave 32, , , ,752 < Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 23 / 44

Approach 1: An example of global seismic inversion inversion field: cp in acoustic wave equation prior mean: PREM (radially symmetric model) truth model: S2RTS (Ritsema et al.

40 Approach 1: An example of global seismic inversion inversion field: cp in acoustic wave equation prior mean: PREM (radially symmetric model) truth model: S2RTS (Ritsema et al.), (laterally heterogeneous) Piecewise-trilinear on same mesh as forward/adjoint 3rd order dg fields dimensions: 1.7 million parameters, 63 million field unknowns Final time: T = 1s with 24 time steps A single forward solve takes 1 minute on 64K Jaguar cores Problem II 8 1 4,842 parameters 67,77 parameters 431,749 parameters 6 eigenvalue number truth, sources (black) Tan Bui-Thanh (UT Austin) MAP, receivers (white) Hessian eigenvalues Large-Scale Uncertainty Quantification Bayesian Inversions 24 / 44

Bui-Thanh, T., Ghattas, O., Martin, J., and Stadler, G., A computational framework for infinite-dimensional Bayesian inverse problems.

41 Approach 1: Uncertainty quantification Details in: C C C 1/2 V r D r V rc 1/2 Bui-Thanh, T., Burstedde, C., Ghattas, O., Martin, J., Stadler, G., and Wilcox, L.C.,Extreme-scale UQ for Bayesian inverse problems governed by PDEs, ACM/IEEE Supercomputing SC12, Gordon Bell Prize Finalist, 212. Bui-Thanh, T., Ghattas, O., Martin, J., and Stadler, G., A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, SIAM Journal on Scientific Computing, 35(6), pp. A2494 A2523, 213. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 25 / 44

Approach 2: MCMC Simulation for Seismic inversion prior distribution posterior distribution posterior sample Use Gaussian approximation as proposal

42 Approach 2: MCMC Simulation for Seismic inversion prior distribution posterior distribution posterior sample Use Gaussian approximation as proposal 15,587 samples, acceptance rate hours on 248 cores 1/1 Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 26 / 44

43 Outline 1 Statistical inverse problem in electromagnetics (Reduced-then-Sample) 2 Ultra-scale Seismic Wave Inversion (Sample-then-Reduce) 3 Big-Data in Inverse problems (if time permits) 4 Summary Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 27 / 44

44 Outline 1 Statistical inverse problem in electromagnetics (Reduced-then-Sample) 2 Ultra-scale Seismic Wave Inversion (Sample-then-Reduce) 3 Big-Data in Inverse problems (if time permits) 4 Summary Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 28 / 44

45 What have we learned? Contents 1 Basic Bayesian probability theory (lecture-oriented) Probability space, random variables, distribution Conditional probability, Bayes formula, prior, likelihood, posterior Construction of likelihood and prior Relation between Bayesian inverse and deterministic inverse problems Monte Carlo and classical limit theorems Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 29 / 44

46 What have we learned? Contents 1 Basic Bayesian probability theory (lecture-oriented) Probability space, random variables, distribution Conditional probability, Bayes formula, prior, likelihood, posterior Construction of likelihood and prior Relation between Bayesian inverse and deterministic inverse problems Monte Carlo and classical limit theorems 2 Advanced topics (lecture-oriented) Advanced MCMC techiniques (Langevin, stochastic Newton, Hamiltonian, randomized MAP) Bayesian inversion in infinite dimensions Discretization-invariant MCMC methods Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 29 / 44

47 What have we learned? Contents 1 Basic Bayesian probability theory (lecture-oriented) Probability space, random variables, distribution Conditional probability, Bayes formula, prior, likelihood, posterior Construction of likelihood and prior Relation between Bayesian inverse and deterministic inverse problems Monte Carlo and classical limit theorems 2 Advanced topics (lecture-oriented) Advanced MCMC techiniques (Langevin, stochastic Newton, Hamiltonian, randomized MAP) Bayesian inversion in infinite dimensions Discretization-invariant MCMC methods 3 Large-scale Bayesian inverse problems (seminar-oriented) Large-scale statistical inverse problems Big-data in large-scale inverse problems Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 29 / 44

48 What else? Contents 1 Polynomial chaos 2 Non-Gaussian priors 3 Approximate MCMCs, non-reversible MCMCs 4 Data assimilation 5 etc Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 3 / 44

49 Conclusions: Reduce-Then-Sample Computational Fluid dynamics: summary CFD is expensive for probabilistic purposes Reduced basis (Reduced-order) modeling seems to be useful Probabilistic analysis with reduced model is cost effective (orders of magnitude less time consuming), yet accurate Computational Electromagnetics: summary 1 Statistical inversion via the Bayesian framework 2 Monte Carlo sampling the posterior in high dimensions is impractical 3 Hessian-based Piecewise Gaussian approximation to the posterior 4 Inverse solution comes with quantifiable uncertainty and more Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 31 / 44

50 Conclusions: Sample-Then-Reduce Full wave form seismic inversion 1 Given noise-corrupted measurements, want to image the earth interior 2 Infinite dimensional Bayesian inference 3 Explore the posterior measure: Derivation of A Γ 1 A 4 Compactness of A Γ 1 A 5 Scalable Discontinuous Galerkin for seismic waves: Scalability of A Main results Able to solve statistical inverse problem with more than one million parameters with more than three orders of magnitude speedup Gaussian approximation seems to be good in this case Inverse solution comes with quantifiable uncertainty and more Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 32 / 44

Future work: Variance-driven adaptivity adaptively refine parameter mesh based on posterior covariance (estimated by the inverse of the data-misfit Hessian) begin with coarse parameter discretization

51 Future work: Variance-driven adaptivity adaptively refine parameter mesh based on posterior covariance (estimated by the inverse of the data-misfit Hessian) begin with coarse parameter discretization relative to wave field mesh locally refine parameter mesh based on variance information allows information contained in the data to drive the medium parametrization Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 33 / 44

52 Future work: Variance-driven adaptivity adaptively refine parameter mesh based on posterior covariance (estimated by the inverse of the data-misfit Hessian) begin with coarse parameter discretization relative to wave field mesh locally refine parameter mesh based on variance information allows information contained in the data to drive the medium parametrization Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 33 / 44

53 Future work: Inverse Electromagnetic Scattering Animated by Greg Abram Blended wing body aircraft Details in: Bui-Thanh, T., Burstedde, C., and Ghattas, O., A discontinuous Galerkin method for electromagnetic wave propagation on h-non-conforming adapted meshes, In preparation, 213. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 34 / 44

Future work: Seismic inversion with real data 5 48 47 3 sec Black Hills 46 45 4 44 43 42 41 3 4 39 12 11 1 9 38-18 -16-14 -12-1

54 Future work: Seismic inversion with real data sec Black Hills Anisotropy Attenuation Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 35 / 44

55 Future work: Infinite dimensional MCMC on function spaces Infinite dimensional scaled stochastic Newton Infinite dimensional randomized maximum likelihood Infinite dimensional hybrid Monte Carlo Infinite dimensional Metropolis-adjusted Langevin method Infinite dimensional random walk Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 36 / 44

56 Future Research Methodology Reduced basis (reduced-order modeling) methods Scalable response surface approach (Infinite dimensional) MCMC on function spaces Optimization-based discontinuous finite element method Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 37 / 44

57 Details of MCMC for seismic inversion 1 Use Gaussian approximation at MAP as a proposal for MCMC 2 Sampling performance for a coarser problem (with 78k parameters): 3 15,587 MCMC samples (each requires 1 forward PDE solve) 4399 samples accepted (28%) 4 Integrated autocorrelation time of about 16 2 ) effective sample size of about 8 5 Total runtime of about 96 hours on 248 cores Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 38 / 44

58 Infinite dimensional Bayesian statistical inference Problem with the usual Bayes formula The usual Bayes theorem π post (u y obs ) π prior (u) π like (y obs u) What are π s? They are probability density with respect to the Lebesgue s measure in R n There is no infinite dimensional Lebesgue s measure! Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 39 / 44

59 Infinite dimensional Bayesian statistical inference Problem with the usual Bayes formula The usual Bayes theorem π post (u y obs ) π prior (u) π like (y obs u) What are π s? They are probability density with respect to the Lebesgue s measure in R n There is no infinite dimensional Lebesgue s measure! π post (u y obs ) π prior (u) π like (y obs u) Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 39 / 44

60 Infinite dimensional Bayesian statistical inference Radon-Nikodym derivative Define µ the prior measure: dµ dλ = π prior, where λ is the Lebesgue measure in R n µ yobs the posterior measure conditional on y obs Rewrite the usual Bayes theorem using Radon-Nikodym derivative dµ yobs ( ( )) (u) π like (y obs u) def = exp Φ y obs, u dµ Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 4 / 44

61 Infinite dimensional Bayesian statistical inference Radon-Nikodym derivative Define µ the prior measure: dµ dλ = π prior, where λ is the Lebesgue measure in R n µ yobs the posterior measure conditional on y obs Rewrite the usual Bayes theorem using Radon-Nikodym derivative dµ yobs ( ( )) (u) π like (y obs u) def = exp Φ y obs, u dµ This formulation is valid for infinite dimensional setting This formulation becomes the usual Bayes theorem in finite dimensions Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 4 / 44

62 Others: Discontinuous Petrov-Galerkin (DPG) Method Optimal convergence p + 1 as opposed to DG with p + 1/2 Details in: Roberts, N., Bui-Thanh, T., and Demkowicz, D., The DPG Method for the Stokes Problem, Computers & Mathematics with Applications, Submitted, 212. Chan, J., Heuer, N., Bui-Thanh, T., and Demkowicz, D., Robust DPG method for convection-dominated diffusion problems II: a natural in flow condition, Computers & Mathematics with Applications, Accepted, 212. Bui-Thanh, T., Demkowicz, L., and Ghattas, O., A Unified Discontinuous Petrov-Galerkin Method and its Analysis for Friedrichs Systems, SIAM Journal on Numerical Analysis, Revised, 212. Bui-Thanh, T., Demkowicz, L., and Ghattas, O., Constructively Well-Posed Approximation Methods with Unity Inf-Sup and Continuity Constants for Partial Differential Equations, Mathematics of Computation, To appear, 212. Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 41 / 44

63 Gauss-Newton Hessian of data misfit with respect to c p D 2 J(c p, c s )(δc p, δ c p, δ c s ) := D e [ T V e D e p δc p + D e T ] S pδc e p dxdt where: V p e =2ρc p ( w) tr(e) S e p = k 2 ρ n [S ]n [ G] + kρ 2 ( ρ + + c ) 2 p [v ][[ ( w ] + kρ 2 ρ + + c p n [S ][[ w ] [v ]n [ G] ) ( + 2k ρ c p tr(e ) n [ G] ) ρ + c + p [ w ] 1 k = ρ c p + ρ + c + p Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 42 / 44

64 Gauss-Newton Hessian of data misfit with respect to c s D 2 J(c p, c s )(δcs, δ c p, δ c s ) := D e [ T V e D e s δcs + where: V s e =4ρc s (E tr(e)i) : 1 ( w + w T ) 2 ( ( S s e = k1ρ 2 (n (n [S ])) n n [ G] )) D e T + k 2 1ρ ( ρ + c s + ) 2 (n (n [v])) (n (n [ w])) ] S sδcs e dxdt + k1ρ 2 ρ + c + s (n (n [S ])) (n (n [ w])) ( ( k1ρ 2 ρ + c + s (n (n [v])) n n [ G] )) + 4k ρ c ( s n E n tr(e ) ) ( n [ G] ) ρ + c + p [ w ] + 4k 1 ρ c ( s n ( n E n )) ( ( ( n n [ G] ))) ρ + c + s [ w] Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 43 / 44

65 State-of-the-art Optimization Technique Subspace Trust Region Interior Reflective Inexact Newton-CG 1 Trust region (deals with ill-conditioning) 2 Interior reflective (deals with bound constraints) 3 Inexact Newton-CG (converges quadratically, prevents oversolving) 4 Uses adjoint method to compute the gradient 5 Computes Hessian-vector product on-the-fly using one forward-like and one adjoint-like solves Efficient optimization solver Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 44 / 44

66 State-of-the-art Optimization Technique Subspace Trust Region Interior Reflective Inexact Newton-CG 1 Trust region (deals with ill-conditioning) 2 Interior reflective (deals with bound constraints) 3 Inexact Newton-CG (converges quadratically, prevents oversolving) 4 Uses adjoint method to compute the gradient 5 Computes Hessian-vector product on-the-fly using one forward-like and one adjoint-like solves Efficient optimization solver The number of Newton steps is numerically observed O(d). d: the number of parameters Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 44 / 44

67 GPU implementation: AMR on CPU, wave prop on GPU Excellent weak scalability on TACC s GPU cluster (collaboration with T. Warburton, Rice) #GPUs #elem mesh transf wave par eff Tflops (s) (s) prop wave (s.p.) Up to 12.3 million 7-th order elements (67 billion unknowns) transf indicates the time to transfer the mesh and other initial data from CPU to GPU memory wave prop is the runtime in µsec per time step per average number of elements per GPU wallclock time about 1 second per time step (meshing and transfer time are completely negligible for realistic simulations) Longhorn = 512 NVIDIA FX 58 GPUs each with 4GB graphics memory and 512 Intel Nehalem quad core processors connected by QDR InfiniBand interconnect Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 45 / 44

68 The Model-Constrained Adaptive Sampling Algorithm 1 Given a reduced basis Φ and initial guess u. Find u = arg max u y(u) y r(u) If y(u ) y r (u ) 2 2 ɛ, then terminate the algorithm. If not, go to the next step. 3 With u = u, solve the full system to compute the state solutions x(u ), which is then used to update the basis Φ. Go to step 1. Error Biot number Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 46 / 44

69 The Model-Constrained Adaptive Sampling Algorithm 1 Given a reduced basis Φ and initial guess u. Find u = arg max u y(u) y r(u) If y(u ) y r (u ) 2 2 ɛ, then terminate the algorithm. If not, go to the next step. 3 With u = u, solve the full system to compute the state solutions x(u ), which is then used to update the basis Φ. Go to step 1. Error Error Biot number Biot number Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 46 / 44

70 The Model-Constrained Adaptive Sampling Algorithm 1 Given a reduced basis Φ and initial guess u. Find u = arg max u y(u) y r(u) If y(u ) y r (u ) 2 2 ɛ, then terminate the algorithm. If not, go to the next step. 3 With u = u, solve the full system to compute the state solutions x(u ), which is then used to update the basis Φ. Go to step 1. Error Error Error 2.5 x Biot number Biot number Biot number Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 46 / 44

71 The Model-Constrained Adaptive Sampling Algorithm 1 Given a reduced basis Φ and initial guess u. Find u = arg max u y(u) y r(u) If y(u ) y r (u ) 2 2 ɛ, then terminate the algorithm. If not, go to the next step. 3 With u = u, solve the full system to compute the state solutions x(u ), which is then used to update the basis Φ. Go to step 1. Error Error Error 2.5 x x Error Biot number Biot number Biot number Biot number Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 46 / 44

72 The Model-Constrained Adaptive Sampling Algorithm 1 Given a reduced basis Φ and initial guess u. Find u = arg max u y(u) y r(u) If y(u ) y r (u ) 2 2 ɛ, then terminate the algorithm. If not, go to the next step. 3 With u = u, solve the full system to compute the state solutions x(u ), which is then used to update the basis Φ. Go to step 1. Error Error Error 2.5 x x x Error Error Biot number Biot number Biot number Biot number Biot number Tan Bui-Thanh (UT Austin) Large-Scale Uncertainty Quantification Bayesian Inversions 46 / 44

A Fresh Look at the Bayes Theorem from Information Theory

A Fresh Look at the Bayes Theorem from Information Theory Tan Bui-Thanh Computational Engineering and Optimization (CEO) Group Department of Aerospace Engineering and Engineering Mechanics Institute for