Steps in Uncertainty Quantification

Size: px

Start display at page:

Download "Steps in Uncertainty Quantification"

Augustine Todd
5 years ago
Views:

1 Steps in Uncertainty Quantification Challenge: How do we do uncertainty quantification for computationally expensive models? Example: We have a computational budget of 5 model evaluations. Bayesian inference and uncertainty propagation require 2, evaluations.

Uncertainty Quantification Challenges Example: MFC model Fourth-order PDE MFC Patch

[k e(e, )E + k 2 " irr (E, )] MFC(x) Bayesian Inference: Took 6 days!

2 Uncertainty Quantification Challenges Example: MFC model Fourth-order PDE MFC = f Capacitor probe M = -c E 2 - c - [k e(e, )E + k 2 " irr (E, )] MFC(x) Bayesian Inference: Took 6 days! Macro-Fiber Composite Problem: c E I.2 5 PDE solutions Solution: Highly efficient surrogate models 38

3 Example: Consider the heat equation with the response Boundary Conditions Initial Conditions Z Z Z Z Surrogate Models: Motivation f (q) u(t, x, y, z)dxdydzdt t x, y, z Notes: Requires approximation of PDE in 3-D What would be a simple surrogate? 39

4 Example: Consider the heat equation with the response Boundary Conditions Initial Conditions Z Z Z Z Surrogate Models: Motivation f (q) u(t, x, y, z)dxdydzdt t Surrogate: Quadratic x, y, z y s (q) =(q -.25) Question: How do you construct a polynomial surrogate? Regression Interpolation

5 Recall: Consider the model with the response Boundary Conditions Initial Conditions Z Z Z Z Surrogate Models f (q) u(t, x, y, z)dxdydzdt t Surrogate: Quadratic x, y, z y s (q) =(q -.25) Question: How do you construct a polynomial surrogate? Interpolation Regression M=7 k=2

6 Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) BIC = k log(m) - 2 log[ (y q)] Likelihood: 2 (y q) = (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 m= Maximize Minimize M=7 k=2 42

Information Criterion (AIC) Likelihood: BIC = k log(m) - 2 log[ (y q)] 2

7 Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) Likelihood: BIC = k log(m) - 2 log[ (y q)] 2 (y q) = (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 Example: Exercise: m= y = exp(.7q +.3q 2 ) Maximize Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate? Minimize q2 q M=7 k=2

8 Notes: Often termed response surface models, emulators, meta-models. Data-Fit Models Constructed via interpolation or regression. Data can consist of high-fidelity simulations or experiments. y = f(q) Example: Steady-state Euler-Bernouilli beam model with PZT patch YI d 4 w dx 4 (x) =k pv pzt (x) Data: Displacement observations Parameter: YI 44

9 Data-Fit Models Example: Steady-state Euler-Bernouilli beam model with PZT patch YI d 4 w dx 4 (x) =k pv pzt (x) Data: Displacement observations Parameter: YI Training points: 5 Polynomial surrogate: 6 th order Bayesian Inference displacement x YI

10 Notes: Often termed response surface models, surrogates, emulators, meta-models. Rely on interpolation or regression. Data-Fit Models Data can consist of high-fidelity simulations or experiments. Common techniques: polynomial models, kriging (Gaussian process regression), orthogonal polynomials. Strategy: Consider high fidelity model y = f (q) with M model evaluations y m = f (q m ), m =,..., M Statistical Model: f s (q): Surrogate for f (q) Surrogate: y K (Q) =f s (Q) = y = f(q) KX k k(q) k= Note: k(q) orthogonal with respect to inner product associated with pdf e.g., Q N(, ): Hermite polynomials Q U(-, ): Legendre polynomials y m = f s (q m )+" m, m =,..., M 46

11 Orthogonal Polynomial Representations Representation: Properties: KX y K (Q) = k k(q) (i) E[y K (Q)] = k= Note: (Q) = implies that (ii) var[y K (Q)] = KX k= 2 k k E[ (Q)] = Z E[ i(q) j(q)] = i(q) j(q) (q)dq = ij i 2 where i = E[ i (Q)] Note: Can be used for: Uncertainty propagation Sobol-based global sensitivity analysis Issue: How does one compute k, k =,..., K? Stochastic Galerkin techniques (Polynomial Chaos Expansion PCE) Nonintrusive PCE (Discrete projection) Stochastic collocation Regression-based methods with sparsity control (Lasso) Note: Methods nonintrusive and treat code as blackbox. 47

12 Orthogonal Polynomial Representations Nonintrusive PCE: to obtain k = Z y(q) k(q) (q)dq k Quadrature: k k Take weighted inner product of y(q) = P k= k k(q) RX y(q r ) k(q r )w r r= Note: (i) Low-dimensional: Tensored -D quadrature rules e.g., Gaussian (ii) Moderate-dimensional: Sparse grid (Smolyak) techniques (iii) High-dimensional: Monte Carlo or quasi-monte Carlo (QMC) techniques Regression-Based Methods with Sparsity Control (Lasso): Solve KX min k - dk 2 subject to k 6 2R K + k= Note: Sample points {q m } M m= 2 R M (K +) where jk = k(q j ) d =[y(q ),..., y(q m )] e.g., SPGL MATLAB Solver for large-scale sparse reconstruction 48

13 Stochastic Collocation Strategy: Consider high fidelity model y = f (q) y m with M model evaluations y m = f (q m ), m =,..., M Collocation Surrogate: MX Y M (q) = y m L m (q) m= where L m (q) is a Lagrange polynomial, which in -D, is represented by L m (q) = MY j= j6=m q - q j q m - q j = (q - q ) (q - q m- )(q - q m+ ) (q - q M ) (q m - q ) (q m - q m- )(q m - q m+ ) (q m - q M ) Note: L m (q j )= jm =, j 6= m, j = m Result: Y M (q m )=y m 49

14 Orthogonal Polynomial Methods for PDE Evolution Model: e.g., = N(u, Q)+F(Q), x 2 D, t 2 [, ) B(u, Q) =G(Q), x t 2 [, ) u(, x, Q) =I(Q), x 2 D Weak Formulation: For all v 2 V Z Z vdx + N(u, Q)S(v)dx = F(Q)vdx D D Z Response: y(t, x) = u(t, x, q) (q)dq Example: 2 y(t,)=u(t, L) = u(, x) =u (x) For all v 2 H (, L) vdx + dv dx = Representation: KX u K (t, x, Q) = u k (t, x) k(q) = k= KX k= j= JX u jk (t) j (x) k(q) Discrete Projection: u k (t, x) RX u(t, x, q r ) k(q r )w r k e.g., Finite elements r= 5

15 Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M.2.8 f(q) q 5

16 Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M.2.8 f(q) q f(q) q 52

17 Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M q j = - cos (j - ) M -, j =,..., M f(q) f(q) q q 2 f(q) q 53

18 Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M q j = - cos (j - ) M -, j =,..., M f(q) f(q) q q.5.8 f(q) 2 4 f(q) q q 54

19 Sparse Grid Techniques Tensored Grids: Exponential growth Sparse Grids: Same accuracy p R` Sparse Grid R Tensored Grid R =(R`) p , > ,9 > ,353,8 >

20 Surrogate Construction: Nuclear Power Plant Design Subchannel Code (COBRA-TF): 33 VUQ parameters reduced to 5 using SA Surrogate: Total pressure drop Kriging (GP) emulator constructed using 5 COBRA-TF runs perturbing 5 active inputs. Use remaining computational budget to evaluate quality of surrogate using postprocessed Dakota outputs. Predicted Total Pressure Drop Sample Quantities 2 Out-of-Sample Validation Out of sample Validation Calculated Total Pressure Drop 2 2 Theoretical Quantities 56

21 Model: ds dt = bn - L S B L apple L + B L - di dt = LS B L apple L + B L + dr = I - br dt db H = I - B H dt db L = B H - B L dt Example: SIR Cholera Model H S H S B H apple H + B H - ( B H apple H + B H - bs + b)i S β L β H I ξ B H χ B L γ δ R Model Parameter Symbol Units Values Rate of drinking B L cholera L.5 week Rate of drinking B H cholera H 7.5 ( ) week # bacteria B L cholera carrying capacity apple L m` 6 # bacteria apple B H cholera carrying capacity apple L H m` 7 Human birth and death rate b week Rate of decay from B H to B L week 5 # bacteria Rate at which infectious individuals 7 spread B H bacteria to water # individuals m` week 7 Death rate of B L cholera Rate of recovery from cholera week week

22 generalized total indices Example: SIR Cholera Model Strategy: Employed collocation and discrete projection with sparse grids to compute time-dependent global sensitivity indices time [weeks] apple L b L H L Conclusion: Sensitive indices : Recovery rate H: Rate of drinking B H cholera apple L : B L carrying capacity; Note apple H = apple L /7 : Rate at which B H bacteria spread generalized Sobol index MC (N mc = 4 ) Alg (N quad = 69) Alg (N quad = 277) Alg 2 (N mc = 5/N kl = 2) Alg 2 (N mc = 5/N kl = 5) Alg 2 (N mc = 5/N kl = 5) L H apple L b L parameter

23 Model: ds dt = bn - L S B L apple L + B L - di dt = LS B L apple L + B L + dr = I - br dt db H = I - B H dt db L = B H - B L dt Example: SIR Cholera Model H S H S B H apple H + B H - ( B H apple H + B H - bs + b)i S β L β H I ξ B H χ B L γ δ R Model Parameter Symbol Units Values Rate of drinking B L cholera L.5 week Rate of drinking B H cholera H 7.5 ( ) week # bacteria B L cholera carrying capacity apple L m` 6 # bacteria apple B H cholera carrying capacity apple L H m` 7 Human birth and death rate b week Rate of decay from B H to B L week 5 # bacteria Rate at which infectious individuals 7 spread B H bacteria to water # individuals m` week 7 Death rate of B L cholera Rate of recovery from cholera week week

24 Steps in Uncertainty Quantification 6

25 6. Quantification of Model Discrepancy Thin Beam Essentially all models are wrong, but some are useful George E.P. Box Example: Thin beam driven by PZT patches Euler-Bernoulli Model: with Z L apple 2 dt = k p V (t) Note: 7 parameters, 32 states Statistical Model: Z x2 x For all dx dx + 2 V Z L apple (x) = hb + p h p b p p (x), YI(x) =YI + Y p I p p (x) ci(x) =ci + c p I p p (x) t i dx y(t i, q) =w(t i, x, q) Y i = y(t i, q)+" i 6 Y i

26 Quantification of Model Discrepancy Thin Beam Example: Good model fit Y i = y(t i, q)+" i Note: Observation errors not iid x 7 Model Fit to Data Mathematical Model y(t i ;q) Data Reference: Additive observation errors Y i = y(t i, q)+ (t i, eq)+" i M.C. Kennedy and A. O Hagan, Journal of the Royal Statistical Society, Series B, 2. Frequency Content Frequency

27 Quantification of Model Discrepancy Thin Beam Example: Good model fit Problem: Observation errors not iid Model Fit to Data Result: Prediction intervals wrong Approaches: GP Model: Inaccurate for extrapolation Control-based approaches: difficult to extrapolate. Problem: correct physics or biology required for extrapolation! Prediction Intervals and Data

Embed uncertainty into latter. T. Oliver, G. Terejanu, C.S. Simmons, R.D. Moser, Comput Meth Appl Mech Eng, 25.

28 Quantification of Model Discrepancy Thin Beam Problem: Measurement errors not iid Result: Prediction intervals wrong Model Fit to Data One Approach: Determine components of model you trust (e.g., conservation laws) and don t trust (e.g., closure relations). Embed uncertainty into latter. T. Oliver, G. Terejanu, C.S. Simmons, R.D. Moser, Comput Meth Appl Mech Eng, SAMSI Program: Model Uncertainty: Mathematical and Statistical (MUMS) Prediction Intervals and Data

29 Quantification of Model Discrepancy Thin Beam Our Solution: Optimize calibration interval Use damping/frequency domain results to guide Displacement (µm) Time (s) Displacement (µm) Time (s) Calibrate on [,] Calibrate on [.25,.25] Note: We have substantially extended calibration regime Displacement (µm) Time (s) Displacement (µm) Time (s) Displacement (µm) Time (s)

Goal is to predict model responses with quantified and reduced uncertainties. Parameter selection is critical to isolate identifiable and influential parameters.

30 Notes: Concluding Remarks UQ requires a synergy between engineering, statistics, and applied mathematics. Model calibration, model selection, uncertainty propagation and experimental design are natural in a Bayesian framework. Goal is to predict model responses with quantified and reduced uncertainties. Parameter selection is critical to isolate identifiable and influential parameters. Surrogate models critical for computationally intensive simulation codes. Codes and packages: Sandia Dakota, R, MATLAB, Python, nanohub. Prediction is very difficult, especially if it s about the future, Niels Bohr. Displacement (µm) Time (s)

31 Books: References R.C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, SIAM, Philadelphia, 24. Incomplete References: Orthogonal Polynomial Methods and High-Dimensional Approximation Theory: M. Babuška, F. Nobile and R. Tempone, Numer. Math., 25 A. Cohen, R. De Vore and C. Schwab, Analysis and Applications, 2 M. Gunzburger, C.G. Webster and G. Zhang, Acta Numerica, 24. O.P. Le Maître and O.M. Knio, Spectral Methods for Uncertainty Quantification, 2 S., Uncertainty Quantification: Theory, Implementation, and Applications, 24. A.L. Teckentrup, P. Jantsch, M. Gunzburger and C.G. Webster, SIAM/ASA J. Uncert. Quant., 25 D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, 2. 67

32 References Incomplete References: Sparse Grids: H-J. Bungartz and M. Griebel, Acta Numerica, 24 M. Gunzburger et al., Water Resources Research, 23. F. Nobile, R. Tempone and C.G. Webster, SIAM J. Num. Anal., 28 Compressed Sensing: A. Chkifra, N. Dexter, H. Tran and C.G. Webster, Mathematics of Computation, Submitted Infinite-Dimensional Bayesian Inference: A.M. Stuart, Acta Numerica 2 T. Bui-Thanh et al., SIAM J. Sci. Comput., 23 S.J. Vollmer, SIAM/ASA J. Uncertainty Quantification, 25. T. Bui-Thanh and Q. Nguyen, Inverse Problems and Imaging, 26, 68

Parameter Selection Techniques and Surrogate Models

Parameter Selection Techniques and Surrogate Models Model Reduction: Will discuss two forms Parameter space reduction Surrogate models to reduce model complexity Input Representation Local Sensitivity