Steps in Uncertainty Quantification Challenge: How do we do uncertainty quantification for computationally expensive models? Example: We have a computational budget of 5 model evaluations. Bayesian inference and uncertainty propagation require 2, evaluations.
Uncertainty Quantification Challenges Example: MFC model Fourth-order PDE MFC Patch @2 w @t 2 - @2 M @x 2 = f Capacitor probe M = -c E I @2 w @x 2 - c DI @3 w @x 2 @t - [k e(e, )E + k 2 " irr (E, )] MFC(x) Bayesian Inference: Took 6 days! Macro-Fiber Composite Problem: c E I.2 5 PDE solutions Solution: Highly efficient surrogate models 38
Example: Consider the heat equation with the response y(q) = @u @t Boundary Conditions Initial Conditions Z Z Z Z Surrogate Models: Motivation = @2 u @x 2 + @2 u @y 2 + @2 u @z 2 + f (q) u(t, x, y, z)dxdydzdt t x, y, z Notes: Requires approximation of PDE in 3-D What would be a simple surrogate? 39
Example: Consider the heat equation with the response y(q) = @u @t Boundary Conditions Initial Conditions Z Z Z Z Surrogate Models: Motivation = @2 u @x 2 + @2 u @y 2 + @2 u @z 2 + f (q) u(t, x, y, z)dxdydzdt t Surrogate: Quadratic x, y, z y s (q) =(q -.25) 2 +.5 Question: How do you construct a polynomial surrogate? Regression Interpolation
Recall: Consider the model with the response y(q) = @u @t Boundary Conditions Initial Conditions Z Z Z Z Surrogate Models = @2 u @x 2 + @2 u @y 2 + @2 u @z 2 + f (q) u(t, x, y, z)dxdydzdt t Surrogate: Quadratic x, y, z y s (q) =(q -.25) 2 +.5 Question: How do you construct a polynomial surrogate? Interpolation Regression M=7 k=2
Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) BIC = k log(m) - 2 log[ (y q)] Likelihood: 2 (y q) = (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 m= Maximize Minimize M=7 k=2 42
Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) Likelihood: BIC = k log(m) - 2 log[ (y q)] 2 (y q) = (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 Example: Exercise: m= y = exp(.7q +.3q 2 ) Maximize Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate? Minimize q2 q M=7 k=2
Notes: Often termed response surface models, emulators, meta-models. Data-Fit Models Constructed via interpolation or regression. Data can consist of high-fidelity simulations or experiments. y = f(q) Example: Steady-state Euler-Bernouilli beam model with PZT patch YI d 4 w dx 4 (x) =k pv pzt (x) Data: Displacement observations Parameter: YI 44
Data-Fit Models Example: Steady-state Euler-Bernouilli beam model with PZT patch YI d 4 w dx 4 (x) =k pv pzt (x) Data: Displacement observations Parameter: YI Training points: 5 Polynomial surrogate: 6 th order Bayesian Inference displacement x YI
Notes: Often termed response surface models, surrogates, emulators, meta-models. Rely on interpolation or regression. Data-Fit Models Data can consist of high-fidelity simulations or experiments. Common techniques: polynomial models, kriging (Gaussian process regression), orthogonal polynomials. Strategy: Consider high fidelity model y = f (q) with M model evaluations y m = f (q m ), m =,..., M Statistical Model: f s (q): Surrogate for f (q) Surrogate: y K (Q) =f s (Q) = y = f(q) KX k k(q) k= Note: k(q) orthogonal with respect to inner product associated with pdf e.g., Q N(, ): Hermite polynomials Q U(-, ): Legendre polynomials y m = f s (q m )+" m, m =,..., M 46
Orthogonal Polynomial Representations Representation: Properties: KX y K (Q) = k k(q) (i) E[y K (Q)] = k= Note: (Q) = implies that (ii) var[y K (Q)] = KX k= 2 k k E[ (Q)] = Z E[ i(q) j(q)] = i(q) j(q) (q)dq = ij i 2 where i = E[ i (Q)] Note: Can be used for: Uncertainty propagation Sobol-based global sensitivity analysis Issue: How does one compute k, k =,..., K? Stochastic Galerkin techniques (Polynomial Chaos Expansion PCE) Nonintrusive PCE (Discrete projection) Stochastic collocation Regression-based methods with sparsity control (Lasso) Note: Methods nonintrusive and treat code as blackbox. 47
Orthogonal Polynomial Representations Nonintrusive PCE: to obtain k = Z y(q) k(q) (q)dq k Quadrature: k k Take weighted inner product of y(q) = P k= k k(q) RX y(q r ) k(q r )w r r= Note: (i) Low-dimensional: Tensored -D quadrature rules e.g., Gaussian (ii) Moderate-dimensional: Sparse grid (Smolyak) techniques (iii) High-dimensional: Monte Carlo or quasi-monte Carlo (QMC) techniques Regression-Based Methods with Sparsity Control (Lasso): Solve KX min k - dk 2 subject to k 6 2R K + k= Note: Sample points {q m } M m= 2 R M (K +) where jk = k(q j ) d =[y(q ),..., y(q m )] e.g., SPGL MATLAB Solver for large-scale sparse reconstruction 48
Stochastic Collocation Strategy: Consider high fidelity model y = f (q) y m with M model evaluations y m = f (q m ), m =,..., M Collocation Surrogate: MX Y M (q) = y m L m (q) m= where L m (q) is a Lagrange polynomial, which in -D, is represented by L m (q) = MY j= j6=m q - q j q m - q j = (q - q ) (q - q m- )(q - q m+ ) (q - q M ) (q m - q ) (q m - q m- )(q m - q m+ ) (q m - q M ) Note: L m (q j )= jm =, j 6= m, j = m Result: Y M (q m )=y m 49
Orthogonal Polynomial Methods for PDE Evolution Model: e.g., thermal-hydraulic equations @u @t = N(u, Q)+F(Q), x 2 D, t 2 [, ) B(u, Q) =G(Q), x 2 @D, t 2 [, ) u(, x, Q) =I(Q), x 2 D Weak Formulation: For all v 2 V Z Z Z @u D @t vdx + N(u, Q)S(v)dx = F(Q)vdx D D Z Response: y(t, x) = u(t, x, q) (q)dq Example: q = @u @t = @2 u @x 2 y(t,)=u(t, L) = u(, x) =u (x) For all v 2 H (, L) Z L @u @t vdx + Z L @u @x dv dx = Representation: KX u K (t, x, Q) = u k (t, x) k(q) = k= KX k= j= JX u jk (t) j (x) k(q) Discrete Projection: u k (t, x) RX u(t, x, q r ) k(q r )w r k e.g., Finite elements r= 5
Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M.2.8 f(q).6.4.2.2.5.5 q 5
Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M.2.8 f(q).6.4.2.2.5 q.5 2 2 f(q) 4 6 8.5.5 q 52
Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M q j = - cos (j - ) M -, j =,..., M f(q).2.8.6.4.2 f(q).8.6.4.2.2.5 q.5 2.5.5 q 2 f(q) 4 6 8.5.5 q 53
Surrogate Models Grid Choice Example: Consider the Runge function f (q) = +25q 2 with points q j = - +(j - ) 2 M, j =,..., M q j = - cos (j - ) M -, j =,..., M f(q).2.8.6.4.2 f(q).8.6.4.2.2.5 q.5 2.5 q.5.8 f(q) 2 4 f(q).6.4 6 8.5.5 q.2.5.5 q 54
Sparse Grid Techniques Tensored Grids: Exponential growth Sparse Grids: Same accuracy p R` Sparse Grid R Tensored Grid R =(R`) p 2 9 29 8 5 9 24 59,49 9 58 > 3 9 5 9 7,9 > 5 47 9,353,8 > 2 95 55
Surrogate Construction: Nuclear Power Plant Design Subchannel Code (COBRA-TF): 33 VUQ parameters reduced to 5 using SA Surrogate: Total pressure drop Kriging (GP) emulator constructed using 5 COBRA-TF runs perturbing 5 active inputs. Use remaining computational budget to evaluate quality of surrogate using postprocessed Dakota outputs. Predicted Total Pressure Drop..5.2.25 Sample Quantities 2 Out-of-Sample Validation Out of sample Validation..5.2.25 Calculated Total Pressure Drop 2 2 Theoretical Quantities 56
Model: ds dt = bn - L S B L apple L + B L - di dt = LS B L apple L + B L + dr = I - br dt db H = I - B H dt db L = B H - B L dt Example: SIR Cholera Model H S H S B H apple H + B H - ( B H apple H + B H - bs + b)i S β L β H I ξ B H χ B L γ δ R Model Parameter Symbol Units Values Rate of drinking B L cholera L.5 week Rate of drinking B H cholera H 7.5 ( ) week # bacteria B L cholera carrying capacity apple L m` 6 # bacteria apple B H cholera carrying capacity apple L H m` 7 Human birth and death rate b week 56 68 Rate of decay from B H to B L week 5 # bacteria Rate at which infectious individuals 7 spread B H bacteria to water # individuals m` week 7 Death rate of B L cholera Rate of recovery from cholera week week 3 7 5 57
generalized total indices Example: SIR Cholera Model Strategy: Employed collocation and discrete projection with sparse grids to compute time-dependent global sensitivity indices..8.6.4.2 2 time [weeks] apple L b L H L Conclusion: Sensitive indices : Recovery rate H: Rate of drinking B H cholera apple L : B L carrying capacity; Note apple H = apple L /7 : Rate at which B H bacteria spread generalized Sobol index.5.4.3.2. MC (N mc = 4 ) Alg (N quad = 69) Alg (N quad = 277) Alg 2 (N mc = 5/N kl = 2) Alg 2 (N mc = 5/N kl = 5) Alg 2 (N mc = 5/N kl = 5) L H apple L b L parameter
Model: ds dt = bn - L S B L apple L + B L - di dt = LS B L apple L + B L + dr = I - br dt db H = I - B H dt db L = B H - B L dt Example: SIR Cholera Model H S H S B H apple H + B H - ( B H apple H + B H - bs + b)i S β L β H I ξ B H χ B L γ δ R Model Parameter Symbol Units Values Rate of drinking B L cholera L.5 week Rate of drinking B H cholera H 7.5 ( ) week # bacteria B L cholera carrying capacity apple L m` 6 # bacteria apple B H cholera carrying capacity apple L H m` 7 Human birth and death rate b week 56 68 Rate of decay from B H to B L week 5 # bacteria Rate at which infectious individuals 7 spread B H bacteria to water # individuals m` week 7 Death rate of B L cholera Rate of recovery from cholera week week 3 7 5 59
Steps in Uncertainty Quantification 6
6. Quantification of Model Discrepancy Thin Beam Essentially all models are wrong, but some are useful George E.P. Box Example: Thin beam driven by PZT patches Euler-Bernoulli Model: with Z L apple (x) @2 w @t + @w 2 dt = k p V (t) Note: 7 parameters, 32 states Statistical Model: Z x2 x For all dx dx + 2 V Z L apple YI(x) @2 w @x 2 + ci(x) @3 w @x 2 @t (x) = hb + p h p b p p (x), YI(x) =YI + Y p I p p (x) ci(x) =ci + c p I p p (x) t i dx y(t i, q) =w(t i, x, q) Y i = y(t i, q)+" i 6 Y i
Quantification of Model Discrepancy Thin Beam Example: Good model fit Y i = y(t i, q)+" i Note: Observation errors not iid x 7 Model Fit to Data Mathematical Model y(t i ;q) Data Reference: Additive observation errors Y i = y(t i, q)+ (t i, eq)+" i M.C. Kennedy and A. O Hagan, Journal of the Royal Statistical Society, Series B, 2. Frequency Content.8.6.4.2 2 4 6 8 2 Frequency
Quantification of Model Discrepancy Thin Beam Example: Good model fit Problem: Observation errors not iid Model Fit to Data Result: Prediction intervals wrong Approaches: GP Model: Inaccurate for extrapolation Control-based approaches: difficult to extrapolate. Problem: correct physics or biology required for extrapolation! Prediction Intervals and Data
Quantification of Model Discrepancy Thin Beam Problem: Measurement errors not iid Result: Prediction intervals wrong Model Fit to Data One Approach: Determine components of model you trust (e.g., conservation laws) and don t trust (e.g., closure relations). Embed uncertainty into latter. T. Oliver, G. Terejanu, C.S. Simmons, R.D. Moser, Comput Meth Appl Mech Eng, 25. 28-9 SAMSI Program: Model Uncertainty: Mathematical and Statistical (MUMS) Prediction Intervals and Data
Quantification of Model Discrepancy Thin Beam Our Solution: Optimize calibration interval Use damping/frequency domain results to guide. 3 2 3 2 Displacement (µm) 2 3 4 2 3 Time (s) Displacement (µm) 2 3 2 3 Time (s) Calibrate on [,] Calibrate on [.25,.25] Note: We have substantially extended calibration regime. 6 5 4 Displacement (µm) 4 2 2 4 6.75.8.85.9.95 Time (s) Displacement (µm) 5 2 2. 2.2 2.3 2.4 2.5 Time (s) Displacement (µm) 3 2 2 3 4 3 3. 3.2 3.3 3.4 3.5 Time (s)
Notes: Concluding Remarks UQ requires a synergy between engineering, statistics, and applied mathematics. Model calibration, model selection, uncertainty propagation and experimental design are natural in a Bayesian framework. Goal is to predict model responses with quantified and reduced uncertainties. Parameter selection is critical to isolate identifiable and influential parameters. Surrogate models critical for computationally intensive simulation codes. Codes and packages: Sandia Dakota, R, MATLAB, Python, nanohub. Prediction is very difficult, especially if it s about the future, Niels Bohr. Displacement (µm) 5 5 2 2. 2.2 2.3 2.4 2.5 Time (s)
Books: References R.C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, SIAM, Philadelphia, 24. Incomplete References: Orthogonal Polynomial Methods and High-Dimensional Approximation Theory: M. Babuška, F. Nobile and R. Tempone, Numer. Math., 25 A. Cohen, R. De Vore and C. Schwab, Analysis and Applications, 2 M. Gunzburger, C.G. Webster and G. Zhang, Acta Numerica, 24. O.P. Le Maître and O.M. Knio, Spectral Methods for Uncertainty Quantification, 2 S., Uncertainty Quantification: Theory, Implementation, and Applications, 24. A.L. Teckentrup, P. Jantsch, M. Gunzburger and C.G. Webster, SIAM/ASA J. Uncert. Quant., 25 D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, 2. 67
References Incomplete References: Sparse Grids: H-J. Bungartz and M. Griebel, Acta Numerica, 24 M. Gunzburger et al., Water Resources Research, 23. F. Nobile, R. Tempone and C.G. Webster, SIAM J. Num. Anal., 28 Compressed Sensing: A. Chkifra, N. Dexter, H. Tran and C.G. Webster, Mathematics of Computation, Submitted Infinite-Dimensional Bayesian Inference: A.M. Stuart, Acta Numerica 2 T. Bui-Thanh et al., SIAM J. Sci. Comput., 23 S.J. Vollmer, SIAM/ASA J. Uncertainty Quantification, 25. T. Bui-Thanh and Q. Nguyen, Inverse Problems and Imaging, 26, 68