UT-LANL DOE/ASCR ISICLES Project PARADICE: Uncertainty Quantification for Large-Scale Ice Sheet Modeling and Simulation.

Transcription

1 UT-LANL DOE/ASCR ISICLES Project PARADICE: Uncertainty Quantification for Large-Scale Ice Sheet Modeling and Simulation Omar Ghattas Representing the rest of the UT-Austin/LANL team: Don Blankenship Tan Bui-Thanh Carsten Burstedde Jim Gattiker Dave Higdon Toby Isaac Charles Jackson James Martin Stephen Price Georg Stadler Lucas Wilcox (Bill Lipscomb ) Institute for Computational Engineering and Sciences, UT-Austin Jackson School of Geosciences, UT-Austin Dept. of Mechanical Engineering, UT-Austin COSIM/Fluid Dynamics Group, Los Alamos National Laboratory Statistical Sciences, Los Alamos National Laboratory 1 / 27

2 Overall project goals Create scalable, parallel, adaptive full Stokes ice sheet simulator equipped with deterministic and statistical inversion capabilities Integrate as an alternative dynamics model with CISM Quantify uncertainties in predictions of ice sheet dynamics in West Antarctica through assimilation of observational data 2 / 27

3 Research challenges Scalable linear solvers for variable-coefficient Stokes Scalable nonlinear solvers for nonlinear rheology Adaptivity to resolve flow transitions and high velocity gradients Inverse methods to identify unknown constitutive parameters, basal boundary conditions, and geothermal heat flux Statistical inference methods to estimate uncertainty in unknowns Assimilation of observational data into models to quantify uncertainties in ice sheet dynamics predictions 3 / 27

4 Overview of current parallel AMR framework for creeping non-newtonian Stokesian flow and transport u = 0 [ ( η(t, u) u + u ) ] pi = ρ 0 [1 α (T T 0 )] g ( ) T ρ 0 c t + u T (k T ) = H(u) [ ( n n η u + u ) ] pi n = 0 on Ω u n = 0 and T = T bc on Ω and T = T inc at t = 0 Variables: T (x, t), u(x, t), p(x, t) temperature, velocity, pressure Parameters: Ra Rayleigh number H(u), η(t, u) heat production rate, viscosity ρ 0, T 0 reference density and temperature k, c thermal conductivity, specific heat g, α gravitational acceleration, coefficient of thermal expansion 4 / 27

5 Mantle rheology ( ) 1 ( ) d q n 1 n η(t, u) = r ε 2n Ea + pv a II exp A C OH nrt η(t, u) effective viscosity d grain size C OH water content in parts per million of silicon ε II second invariant of strain rate tensor E a the activation energy p pressure V a activation volume R universal gas constant T temperature A, n, r, q parameters 5 / 27

6 Global mantle flow AMR simulations with realistic rheology and 1km local resolution at plate boundaries using Rhea 6 / 27

7 Global mantle flow AMR simulations with realistic rheology and 1km local resolution at plate boundaries using Rhea 7 / 27

8 p4est: Adaptive parallel forests of octrees library Proc 0 Proc 1 Proc / 27

9 mangll: High-order spectral element discretization library tailored to forest-of-octrees mesh library (p4est) Hexahedral elements with 2:1 nonconforming adapted faces Arbitrary order spectral element basis functions (nodal, GLL) Continuous and discontinuous Galerkin approximations Isoparametric or piecewise diffeomorphic geometry mapping 9 / 27

10 Weak scalability of AMR for advection on spherical shell Excellent scalability from 12 to 220,000 cores 1 Normalized work per core per total run time Parallel efficiency K 32K 65K 130K 220K Number of CPU cores Figure: mesh is adapted every 32 time steps; explicit time stepping; discontinuous Galerkin, 3rd order spectral elements, 3200 elements/core (45B DOF on 220K cores); ORNL Jaguar system 10 / 27

11 Weak scalability of AMR for advection on spherical shell AMR consumes < 30% of overall time (unoptimized implementation) AMR and projection Time integration Percentage of total run time K 32K 65K 130K 220K Number of CPU cores Figure: mesh is adapted every 32 time steps; explicit time stepping; discontinuous Galerkin, 3rd order spectral elements, 3200 elements/core (45B DOF on 220K cores); ORNL Jaguar system 11 / 27

12 Rhea: adaptive mantle convection code Summary of discretization and solution Trilinear FEM for temperature, velocity, and pressure Conforming approximation by algebraic elimination of hanging nodes FEM stabilization: Streamline Upwind/Petrov Galerkin (SUPG) for advection-diffusion system Polynomial pressure projection for stabilization of Stokes equation (Dohrmann and Bochev) Explicit integration of energy equation decouples temperature update from nonlinear Stokes solve Nonlinear Stokes solver: lagged-viscosity (Picard) iteration Linear Stokes solver: MINRES iteration with block preconditioner based on AMG V-cycle and inverse viscosity mass matrix approximation of Schur complement 12 / 27

13 MINRES preconditioner for linear Stokes solve Block factorization of Stokes system: ( ) ( ) ( ) ( ) A B I 0 A 0 I A = 1 B B C BA 1 I 0 (BA 1 B + C) 0 I where A is discrete viscous operator, B is discrete divergence, and C is pressure stabilization Suggests preconditioner of form: «A 0 P = with S = BA 1 B + C 0 S Approximate inverse of A with one V-cycle of Algebraic Multigrid (e.g. ML or BoomerAMG) Approximate S with diagonal inverse-viscosity lumped-mass matrix M (spectrally equivalent in isoviscous case) 13 / 27

14 Independence of solver w.r.t. viscosity variation Using BoomerAMG on Cartesian geometry µ min µ max # MINRES AMG setup solve time per iterations time (s) iteration (s) 1.00e e e e e e e Figure: 216M unknowns, 512 cores 14 / 27

15 Weak scalability of Stokes iterative solver in Rhea #cores #elem/ core #elem #dofs #iter setup time [s] matvecs & inner prod. [s] Vcycle time [s] K 2.68M M 18.7M M 146M M 1.26B M 2.51B Mid-ocean ridge benchmark problem Weak scaling with 5000 elements per core on ORNL Jaguar system Mesh contains elements over a range of three refinements Viscosity varies over an order of magnitude Number of MINRES iterations for decrease of residual by factor of 10 4 AMG setup and V-cycle time based on ML from Trilinos with RCB/Zoltan repartitioning within multigrid hierarchy Matvec/inner product time includes all MINRES time other than preconditioner Note that in practice, AMG setup is amortized over numerous linear solves 15 / 27

16 Inverse ice sheet modeling I Observations: internal layering structure (ice-penetrating radar), surface flow velocity (InSAR), age (from ice cores), surface elevation (altimetry) I Inversion variables: constitutive parameters, basal boundary Background: What affects ice streams? conditions, geothermal heat flux, initial temperature Ice Str ice Bindshadler Ice Stream Margin (N. Nereson) 16 / 27

17 Volume = {52}, InverseYear ice=sheet {2006}}modeling, cont. 17 / 27

18 Inverse ice sheet modeling, cont. UTIG ice penetrating radar 18 / 27

19 A general ice sheet inverse problem minimize F(n, A 0, Q, q B, β B) := u u obs ΓF S + z z obs ΓF S + φ φ obs Ω + n n pr Ω + A 0 A pr 0 Ω + Q Q pr Ω + q q pr ΓB + β β pr ΓB subject to: η(t, u) = 1 2 A 1 n h i u = 0, η(t, u) ( u + u T ) Ip = ρg 1 n ε 2n II, ε II = 1 2 tr( ε2 ), ε = 1 2 ( u+ ut ), A = A 0 exp Q «RT Dz Dt Γ F S = a, σn Γ F S = 0, u n Γ B = 0, n n σn+βbn n u Γ B = 0 ρcu T (K T ) = η tr( ε 2 ), T ΓF S = T F S, K T n ΓB = q B ( φ φ) 1 2 = 1 u, φ Γ F S = 0 19 / 27

20 Example: Inverse problem for parameter n subject to: minimize F := u u obs + n u = 0 [η(t, u) ( u + u T ) Ip ] = ρg ( ) T ρc t + u T (K T ) = η tr( ε 2 ) η(t, u) = 1 2 A 1 n ε II = 1 2 tr( ε2 ) ε 1 n 2n II ε = 1 2 ( u + ut ) ( A = A 0 exp Q ) RT 20 / 27

21 Optimality conditions state equations: u = 0 h i η(t, u) ( u + u T ) Ip = ρg «T ρc t + u T (K T ) = η tr( ε 2 ) adjoint equations: v = D pf u + u T : v + v T = h i ρc S T + S D uη tr( ε 2 ) Sη u + u T D uf «ρc (K S) S D T η tr( ε 2 ) = h i η v + v T Iq + Duη 2 S t + u S D T η 2 u + u T : v + v T D T F control equation (e.g. for parameter n): Z t1» 1 u + u T : v + v T S tr( ε 2 ) D nη D nf= 0 t / 27

22 Mathematical and computational issues encountered in deterministic inversion Beyond generic inverse solver issues, the following must be addressed in the context of the inverse ice sheet problem: Gradient-consistent adjoint discretization schemes Inexact Newton-CG for the inverse problem Hessian preconditioning, multilevel methods Appropriate regularization 22 / 27

23 Bayesian inference for inverse problem Bayesian framework for statistical inverse problem: when data and/or model have uncertainties, solution of inverse problem expressed as a posterior probability density function Central challenge: for inverse problems characterized by high-dimensional parameter spaces, method of choice is to sample the posterior density using Markov chain Monte Carlo (MCMC) For inverse problems characterized by expensive forward simulations, contemporary MCMC methods become prohibitive Intractability of MCMC methods for large-scale statistical inverse problems can be traced to their black-box treatment of the parameter-to-observable map (the forward code) Goal: develop methods that exploit the structure of the parameter-to-observation map (including its derivatives), as has been done successfully in deterministic PDE-constrained optimization Hessian-informed Gaussian process response surface approximation Hessian-preconditioned Langevin methods 23 / 27

24 Bayesian formulation of statistical inverse problem Given: π pr (x) := prior p.d.f. of model parameters x π obs (y) := prior p.d.f. of the observables y π model (y x) := conditional p.d.f. relating y and x Then posterior p.d.f. of model parameters is given by: π post (x) def = π post (x y obs ) π obs (y)π model (y x) π pr (x) dy µ(y) Y π pr (x) π(y obs x) From A. Tarantola, Inverse Problem Theory, SIAM, / 27

25 Gaussian additive noise Given the parameter-to-observable map y = f(x), a common noise model is Gaussian additive noise: y obs = f(x) + ε, ε N (0, Γ noise ) If the prior is taken as Gaussian with mean x pr and covariance Γ pr, then the posterior can be written ( ) π post (x) exp 1 2 f(x) y obs 2 1 Γ 1 2 x x pr 2 Γ 1 noise pr Note that most likely point is given by def x MAP = arg max π post (x) x 1 = arg min x 2 f(x) y obs Γ 1 2 x x pr 2 Γ 1 noise pr This is an (appropriately weighted) deterministic inverse problem! 25 / 27

26 Gaussian additive noise, linear inverse problem Suppose further the parameter-to-observable map is linear, i.e. y = F x Then the posterior can be written π post (x) exp ( 1 2 F x y obs 2 Γ 1 noise The posterior is then Gaussian with x N (x MAP, Γ post ) 1 2 x x pr 2 Γ 1 pr The covariance is the inverse Hessian of the negative log posterior: Γ 1 post = F T ΓnoiseF 1 + Γ 1 pr = 2 x( log π post ) ) I.e., the covariance is given by the inverse Hessian of the regularized misfit function that is minimized by deterministic methods 26 / 27

27 Summary of mathematical/computational research agenda Overall goal: Scalable, parallel, adaptive full Stokes ice sheet simulator equipped with deterministic and statistical inversion capabilities Base level enhancements to existing nonlinear Stokes code free surface basal boundary conditions Further forward solver enhancements (as needed) high order spectral element DG/CG discretization fully implicit time integration full Newton solver Deterministic inversion gradient-consistent/adjoint-appropriate discretization globalization of inexact Newton-CG solver multilevel Hessian preconditioner regularization Statistical inversion reduce-then-sample: Gaussian process response surface methods sample-then-reduce: Langevin-based proposals for MCMC 27 / 27