Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems

Transcription

1 Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems Krzysztof Fidkowski, David Galbally*, Karen Willcox* (*MIT) Computational Aerospace Sciences Seminar Aerospace Engineering Department University of Michigan October 3, 2008

2 Outline 1 Forward and Inverse Problems 2 Statistical Inverse Problem 3 Nonlinear Projection-Based Model Reduction 4 Diffusion Flame Application in 2D and 3D

3 Forward Problem µ 2 Given µ y Calculate y Model µ 1 R(u,µ) = 0 y = y(u) Output index µ = parameter vector [µ 1, µ 2 ] y = output vector u = state vector R(u, µ) = model equations

4 Example Model: Finite element discretization of a scalar convection-diffusionreaction equation; scalar = fuel concentration Sample fuel concentration contours µ = reaction rate parameters y = average fuel concentrations at cut-planes u = finite element solution for fuel concentration

5 Inverse Problem Given fuel concentrations, determine reaction rate parameters µ 2 Calculate µ y Given y Model Parameters R(u,µ) = 0 y = y(u) µ 2 Output index Other applications: Medical imaging Circuit identification Model fitting Geophysics

6 Deterministic Inverse Solution µ 2 Calculate µ y Given ȳ Model D µ 1 Output index Determine the best value of the parameter vector: µ = arg min µ y(µ) ȳ 2 (minimization problem) subject to R(u; µ) = 0, (model equations) y(µ) y(u(µ)), µ D. (a priori knowledge)

7 Deterministic Inverse Solution (ctd.) Shortcomings: Experimental errors not included No uncertainty quantification for the best estimate µ The inverse problem may be ill-posed: no unique solution µ µ sensitive to small perturbations in ȳ Practical solution is some form of regularization, for example: µ = arg min µ y(µ) ȳ 2 + β µ 2 β = regularization parameter

8 Statistical Inverse Solution µ 2 Calculate posterior PDF y Given ȳ and ǫ Model ǫ i D µ 1 Output index For example, ǫ = normally-distributed measurement errors, each with standard deviation σ. With this measurement error, likelihood function is: [ p(ȳ µ) exp 1 ] 2σ 2(ȳ y(µ))t (ȳ y(µ)) = probability of measuring ȳ given µ

9 Posterior Probability Distribution Using Bayes theorem: p(µ ȳ) = 1 p(ȳ) p(ȳ µ)p(µ), so that the posterior PDF is (assuming a uniform prior p(µ)) { exp [ 1 (ȳ y(µ)) T (ȳ y(µ)) ], if µ D p(µ ȳ) 2σ 2 0, otherwise. µ 2 p(µ ȳ) The posterior PDF is inferred from the measured outputs, ȳ, and a model for the measurement error, σ, using y(µ). D µ 1 How do we describe p(µ ȳ)?

10 MCMC Sampling Use Markov Chain Monte Carlo (MCMC) to sample the posterior PDF Take a random walk in parameter space Generate sequence: µ 1, µ 2,... p(µ 2 ȳ) µ 2 Taking a step given µ = µ i : Pick µ from a proposal distribution, q Accept µ (i.e. µ i+1 = µ ) with probability: [ α(µ µ) = min 1, p(µ ȳ)q(µ ] µ) p(µ ȳ)q(µ µ, ) Otherwise reject it: (µ i+1 = µ) µ 2 p(µ 1 ȳ) q(µ µ) µ µ µ 1 µ 1

11 The Proposal Distribution, q(µ µ) Choice of q(µ µ) governs exploration of the parameter space, and affects the acceptance probability α. 1. Uniform box Size = [ 1, 2 ] centered at µ q(µ µ) = q(µ µ ) = const. Inefficient for anisotropic posteriors µ 2 Current sample 1 2 Uniform proposal boundary 2. Stretched ellipse [ q(µ µ) exp 1 ] 2δ 2(µ µ) T H(µ)(µ µ) H(µ) = 1 [ ] y T 2σ 2 µ (µ) y µ (µ) Posterior PDF contours µ 1 δ = step-size param y via FD µ

12 Sampling Statistics N m = number of MCMC samples µ i, i = 1..N m are drawn from the posterior probability distribution for N m, expect convergence to the actual probability distribution of µ for finite N m, can only estimate statistics Estimator of a statistical quantity: f 1 N m N m i=1 g(µ i ) mean of µ j g(µ) = µ j variance of µ j g(µ) = (µ j µ j ) 2 ( µ j = sample mean) N m is usually tens of thousands Each evaluation of acceptance probability requires a forward run Cost becomes prohibitive for large simulations

13 Model Reduction Goals: Create a computationally inexpensive emulator of the forward simulation Require accuracy for µ D Retain physics of the problem Take into account non-linearities Assumption For µ D, solution u resides in a low dimensional manifold i.e. can represent it well using n N degrees of freedom.

14 Model Reduction Using Linear Projection N = # unknowns in full system, ( millions) n = # unknowns in reduced system, ( 100) 1 u = Au + Bµ N u Φu r, 1 n Φ T Φ = I n n n N = A + B N = Φ Columns = basis vectors Multiply original system by Φ T to obtain the reduced system: u r = } Φ T {{ AΦ} u r + }{{} Φ T B µ B r A r 1 n n = A r + B r Can precompute A r and B r System is of size n No order N operations to run the reduced system

15 Model Reduction for Nonlinear Systems u = f(u, µ), f(, µ) = nonlinear function Multiplying the original system by Φ T we obtain a reduced system : n 1 u r = Φ T f(u, µ) = N Φ T f( Φ,µ) n unknowns in reduces system but... Cannot precompute any matrix products because of f Need N nonlinearity evaluations this will dominate the cost!

16 Nonlinearity Expansion Key assumption: f(u, µ) resides in a low manifold of dimension m n. f(u, µ) Φ f f r, f r R m N = Substituting into the reduced system: f(u,µ) 1 m Φ f m f r Columns = basis vectors u r = Φ} T {{ Φ} f n m f r (N not present) But, evaluating f r directly still involves N: f r = (Φ f ) T f(u, µ) }{{}}{{} m N N 1 (order N dependent)

17 Masked Projection Compute f r = nonlinearity expansion coefficients approximately: f(u, µ) Φ f f r Zf(u, µ) ZΦ f f r f r (ZΦ f ) 1 Zf(u, µ) f(u, µ) Φ f (ZΦ f ) 1 Zf(u, µ) }{{} Ψ R N m Z is an m N mask matrix Mostly zeros Ones in columns where f is to be evaluated Z = m 1 N 1 1 Ψ = Φ f (ZΦ f ) 1 can be precomputed Zf(u, µ) consists of m evaluations of the nonlinearity Similar to gappy POD [Everson Sirovicz, 1995]

18 Comparison to Direct Projection Direct Projection Masked Projection f(u,µ) 1 = Φ f m f r Zf(u,µ) 1 ZΦ f m f r N = N = m N N 1 = m I 1 m m = (Φ f ) T f(u,µ) = f r Zf(u,µ) ZΦ f f r

19 Reduced Nonlinear System Using f(u, µ) Ψ f f(zu, µ), u r = Φ T f(u, µ) m Φ T Ψ f }{{} E r R n m f(zu, µ) n = n E r m Steps Form Φ and Ψ f basis matrices by, for example, POD on a set of snapshots Choose a mask Z and calculate Ψ = Φ f (ZΦ f ) 1 Calculate E r = Φ T Ψ f offline Each forward solve of reduced model now involves only m n nonlinearity evaluations

20 Choosing a Mask, Z Accuracy of reduced model depends on Z Option 1: Choose Z to minimize cond(zφ f ) [Willcox, 2006] Option 2: Choose Z to minimize error between the masked projection and the full projection of K snaphsots, ξ f k : Z = arg min Z K (Φ f ) T ξ f k (ZΦf ) 1 Zξ f k 2 2 k=1 BPIM = Best Points Interpolation Method [Nguyen et al, 2007] Option 3: Choose i + 1st mask point recursively as the index where the error between Φ f i+1 and its reconstruction using the first i basis vectors is maximum. EIM = Empirical Interpolation Points Method [Nguyen et al, 2007]

21 Convection-Diffusion-Reaction Equations u = scalar fuel concentration (Uu) (ν u) + f(u, µ) = 0 in Ω, u = u D on Ω D, u n = 0 on Ω\ Ω D, u = fuel concentration U = velocity (constant) ν = diffusion coefficient (constant) Nonlinear reaction term: f(u, µ) = Au(c u)e E d u, µ = (ln(a), E)

22 Combustor Model 18mm 3mm Fuel Oxidizer z 1mm 9mm y 3mm x 1mm Measurement Planes

23 Finite Element Discretization 2D: Streamwise Upwind Petrov Galerkin (SUPG) 3D: Discontinuous Galerkin (DG) General discrete form (N unknowns, M nonlinearity evaluations): R(u; µ) = R 0 + Au + E f(du, µ) = 0 D interpolates u to M quadrature points E sums up the nonlinear evaluations Reduced model (n unknowns, m nonlinearity evaluations): R r + A r u r + E r f(d r u r, µ) = 0 E r = Φ T EΨ R n m D r = ZDΦ R m n

24 2D Reduced Model Performance Basis constructed from K = 196 snapshots. Ξ test = test grid in parameter space Average relative error: ε rel = mean µ Ξ test y(µ) y r (µ) y(µ) n m ε rel Online time E E E E E E E E E E 5 Online time is relative to FEM solution ε rel m = 10 m = 15 m = 25 m = 40 m = n n = 40, m = 50 field comparison with full order solution

25 2D Reduced Model Basis Vectors Obtained by Proper Orthogonal Decomposition (Karhunen Loève expansion) of 196 snapshots in a grid in parameter space. First four modes: Mode 1 Mode 2 Mode 3 Mode 4

26 2D Mask Points For m = 15, the mask Z R m N contains 15 nonzero entries. These are the points at which the nonlinear term is evaluated. [Galbally 2005]

27 2D Inverse Problem Results µ Generated ȳ with FEM model and σ = 1.5% measurement error Goal: determine PDF of parameters µ 1 = log(a ) and µ 2 = E Constructed a Markov chain of size N m = 50, 000 with the n = 40, m = 50 reduced model Frequency µ 1 Frequency ROM, samples FE, samples ROM, samples µ MCMC Updates x mean(µ 1 ) mean(µ 2 ) µ MCMC Updates x

28 3D Reduced Model Performance Full-order FEM model: 8.5 million unknowns (13h CPU time) Basis constructed from K = 169 snapshots. Ξ test = test grid in parameter space ε rel 10 Reduced model:.1s CPU time m=10 m=15 m=25 m=40 m=50 n Finite element sol. (8.5 million unknowns) n = 40, m = 50 reduced model (EIM)

29 3D Reduced Model Basis Vectors POD of 169 snapshots in a grid in parameter space. First four modes: Mode 1 Mode 2 Mode 3 Mode 4

30 3D Inverse Problem Results Generated ȳ with FEM model and σ = 1.5% measurement error Goal: determine PDF of parameters µ 1 = log(a ) and µ 2 = E Constructed a Markov chain of size N m = 50, 000 with the n = 40, m = 50 reduced model Frequency Frequency µ µ MCMC Updates x MCMC Updates x µ µ 2 Uniform proposal distribution Acceptance rate = 3.4% (low)

31 Posterior Anisotropy Low acceptance rate of uniform proposal attributed to anisotropy in the posterior PDF, p(y µ) : E Uniform proposal region ln(a) Improve acceptance rate via a stretched-gaussian proposal

32 3D Inverse Problem Results, Stretched Proposal Used δ = 1.5 for the dimensionless step-size parameter Finite differencing for y/ µ Acceptence rate now 25% Only needed N m = 5, 000 samples for same statistics Frequency Frequency µ µ MCMC Updates µ µ 2 Note: MCMC runs with full-order model are prohibitive (13h CPU per forward solve) MCMC Updates

33 Summary and Conclusions Presented a nonlinear model reduction technique in a projection framework Built on previous work in gappy POD, missing point estimation, masked projection, coefficient-function approximation Applied reduction to a parameter estimation problem in a Bayesian inference setting Vast speedup of reduced model makes such an inverse problem solution possible Additional work: Model-constrained adaptive sampling to generate snapshots as number of parameters is increased Quantification of model reduction errors on statistics of interest