The Role of Uncertainty Quantification in Model Verification and Validation. Part 6 NESSUS Exercise Review and Model Calibration

Transcription

1 The Role of Uncertainty Quantification in Model Verification and Validation Part 6 NESSUS Exercise Review and Model Calibration 1

2 Supersolvus Problem Statement 2

3 Probabilistic Analysis Definition 3

4 CDF Results Visualization 4

5 Global Sensitivity Analysis Definition 5

6 Global Sensitivity Visualization 6

7 Exercise Compute the cumulative distribution function for the supersolvus yield strength at 600 o F. use file uq4mm/riha/supersolvus-pre-cal.dat Assume a normal distribution for temperature with a 5 o F standard deviation Analysis type: "Full CDF" Analysis method: Monte Carlo using 10,000 samples Save file as uq4mm/riha/supersolvus-pre-cal_600.dat 1.3 Compare CDF for 1200 o F yield strength. Does the variation change? 7

8 Supersolvus Deterministic Sensitivity Studies Ranges defined in standard normal units of the PDF (similar to standard deviations) All Variables APB0 removed 8

9 Uncertainty Influence at Different Temperatures (Supersolvus) T=1200F T=1200F T=600F T=200F T=600F T=200F 9

10 The Role of Uncertainty Quantification in Model Verification and Validation Model Parameter Calibration 10

11 Model Calibration Motivation Want model predictions to be consistent with test data Predictions should take account of all available information Often model inputs are difficult to characterize May be difficult or expensive to test May not have a distinct physical interpretation Bayesian and other statistical approaches can help quantify uncertainty and confidence based on amount of data and parameter sensitivity Also learn about behavior, limitations, problems with model; or problems with experimental data 11

12 Bayesian analysis for model calibration There may be multiple solutions that are comparable in terms of calibrating the model Because of noise in experimental data, there is also uncertainty associated with these solutions Bayesian analysis quantifies this uncertainty in terms of a posterior distribution for θ Also allows incorporation of prior information about θ 12

13 Calibration Approach Analyzed calibration to supersolvus experimental data Objectives: a) Estimate calibration parameters: Apb0, Qapb b) Estimate residual standard deviation: σ c) Analyze behavior of residuals d) Characterize uncertainty in calibration parameters e) Predict held-back observations, with uncertainty ( x,θ) The model: Predicted Yield Strength = G Control variables (x): TempF, GrSiz, SySiz, SyVf Calibration parameters (θ): Apb0, Qapb Calibration formulation: y i ( x ) i, + ε ( ) i ε ~ N 0,σ = G θ i 13

14 Supersolvus Dataset CoolRate TempF GrSiz PySiz PyVf SySiz SyVf YieldStress E E E E E E E E E E E E E E E E E E E E Total of 20 experimental observations Held back 10 points (highlighted in yellow) for validation Used remaining 10 points for model calibration to tune Apb0 and Qapb 14

15 Calibration Results E[ Apb0 ] = 284 For a point estimate, we could look at either posterior mean or mode (mode would give us least-squares solution, under uniform prior) E[ Qapb] =

16 Prediction of calibration data Bayesian predictions without discrepancy For Calibration data: Max Error: 4.6% Avg Abs Error: 2.16% RMSE: 2.6% Coverage at 95% level: 100% Notes All inputs set to expmt values. Uncertainty in Apb0 and Qapb Percentage error defined as: 100 * (Expmt Predicted) / Expmt Posterior predictive showing mean and 95% prediction intervals 16

17 Model Parameter Uncertainty after Calibration Supersolvus Variable Description Variation PDF Temp Material environmental temperature Estimated Normal(1200,5) GrSize Grain size ± 2.5% Normal(32.5,2.7) PySiz Primary γ size ± 2.5% - PyVf Primary γ V f ± 30% 0.0 SySize Secondary γ size ± 10% Normal(0.29,0.01) SyVf Secondary γ V f ± 10% Normal(0.51, 0.02) APB0 Anti-phase boundary energy ± 10% QAPB Anti-phase boundary energy temperature dependence estimated Normal(284.2,6) Uniform(234, 286) Truncated Weibul (12, 10330, [6000, 10000]) Uniform(9000, 10000) 17

18 Exercise Update uncertainty definitions for supersolvus model use file uq4mm/riha/supersolvus-pre-cal.dat and save as supersolvus-post-cal.dat Analysis type: "Full CDF" Analysis method: Monte Carlo using 10,000 samples 2.2 Compare pre- and post-calibration predicted uncertainty 2.3 Perform global sensitivity analysis and compare preand post sensitivities. Does the importance change? 2.4 How can we further reduce uncertainty? 18