Uncertainty Quantification and Validation Using RAVEN. A. Alfonsi, C. Rabiti. Risk-Informed Safety Margin Characterization.

Transcription

1 Risk-Informed Safety Margin Characterization Uncertainty Quantification and Validation Using RAVEN A. Alfonsi, C. Rabiti North Carolina State University, Raleigh 06/28/2017

2 Assumptions Numerical error negligible (mostly reducible) Verification has been performed (code is bug free) All uncertain parameters are accessible (eases sampling needs) All uncertain parameter distributions are known (otherwise Bayesian)

3 Validation Process Select a number of experiments Perform uncertainty propagation for all experiments (UQ) Compare simulation with experiments (Validation versus experiments) Determine the uncertainties in target prediction (extrapolation) RAVEN current capabilities covers: Uncertainties Quantification (mature) Validation vs. experiments (initial) Extrapolation (planned)

4 UQ

5 UQ Process Goal: determining the Probability Distribution Function (PDF) of the Figure of Merits (FOM) Select the right sampler: Number of variables Non linearity of the model Sample the model according sampler chosen and distributions Analyze the FOM dispersion (mean, sigma etc.)

6 Choice of the Sampling strategy Medium/low Number of variables High Non Linearity High Random and quasi random Samplers Low Surrogate building samplers Surrogate If possible Statistical post processing Direct extraction of statistical moments

7 Sampling strategies RAVEN supports many forward samplers Monte Carlo Grids: equal-spaced in probability equal-spaced in value mixed (probability, custom, value) Custom (used provided values/probability) Stratified (LHS type) equal-spaced in probability equal-spaced in value mixed (probability, custom, value) Custom (used provided values/probability) Generalized stochastic collocation polynomial chaos A different sampling strategies can be associated to each variable separately

8 Sampling strategies (cont.) Factorial Designs: General Full Factorial (grid) 2-Level Fractional-Factorial Plackett-Burman Response Surface Designs: Box-Behnken Central Composite Central Composite Box-Behnken

9 Standard Distributions (CROW) Most common used 1D distributions Probability Distribution Function Truncated Form Available Probability Distribution Function Truncated Form Available Bernoulli No Laplace Yes Beta Yes Logistic Yes Binomial No Lognormal Yes Categorical No Normal Yes Custom1D No Poisson No Exponential Yes Triangular Yes Gamma Yes Uniform Yes Geometric No Weibull Yes

10 N-Dimensional Distributions (CROW) Import from file for custom N-dimensional distributions: N-dimensional splines on Cartesian grids Inverse weight interpolation Micro sphere interpolation Sampling of N-dimensional distributions by not biased random inversion

11 Available Surrogate Models Nearest neighbor (KD-Tree based) Support vector machine: Polynomial kernel Gaussian Kernels Radial basis functions Micro Sphere Inverse Weight N-Dimensional spline Gaussian process Polynomial (stochastic and not) Linear regressors Many more (raven.inl.gov) Ensemble models

12 A Road Map for Collocation Methods ~Tens of parameters Polynomial Assumption Not linear but continuous Looking for FOM distribution Black box approach Modal expansion Generalize the development Probability weighted Original space Probability weighted response SCgPC HDMR Legendre expansion Optimizing resources Adaptive Adaptive Decomposing the variance

13 Stochastic Collocation Generalized Polynomial Chaos SCgPC and Sobolev Indexes Based Usually weighted by the probability Full SCgPC (~10) A priori knowledge of the degree of the function is imposed Full generalized Sobolev decomposition (~10) A priori knowledge of the degree of the function is imposed Sparse grid (~10) Known separability is required Adaptive SCgPC (~100) Separability and almost linearity improves performance Adaptive generalized Sobolev decomposition (~100) Separability and almost linearity improves performance

14 Grid Filling

15 Statistical Post Processing Statistical characterization of the output (uncertainty propagation) Mean Sigma Skewness (asymmetry) Kurtosis (more/less peaked than a standard normal) Input/output relationship (ranking/sensitivity) Correlation matrix Covariance matrix Sensitivity matrix (multidimensional linear regression) Normalized sensitivity matrix (% change of the response / % change in the answer)

16 Example: Bison + RAVEN Power History Power Spike J. Cogliati, J. Chen, J. Patel, D. Mandelli, D. Maljovec, A. Alfonsi, P. Talbot, C. Wang, C. Rabiti Time-Dependent Data Mining in RAVEN INL/EXT Geometry

17 Uncertainty Input

18 Figure Of Merits

19 Uncertainty On FOM 7000 MC Runs 128 BISON simulations simultaneously with each using 16 MPI processes (total of 2048 cores simultaneously used)

20 Contribution to Dispersion: Covariance

21 Validation

22 A Probabilistic Reading of Experimental Data Experimental Data Input Variables (initial and boundary condition) Experimental Measurement (Figures of Merit (FOM)) Uncertainty effect both inputs and readings Probability distribution of the input space Probability distribution of the experimental readings (FOMs)

23 then the Comparison Probabilistic Input Model Experimen t Probabilistic Code Output Probabilistic Experimental Reading ?

24 The Process Input Distribution Sampling Strategy Binning density function estimators Grid based reconstruction ROMs Analytical expression of FOMs cdf Comparative metrics

25 The goal is to achieve a numerical representation a probability density function of a set of point in the output space The less distorting representation is generate by the binning (histogram) The number of bins and its boundaries should be choose to regularize the function without altering its meaning Binning algorithms Square root: Sturge s Formula: Reconstruction of the FOM Distribution

26 Next Step. How we compare the simulation to the experiment?? Mean and Sigma are not enough to compare the model output distributions to the experimental reading distributions The metric should be more extensive and to consider the whole PDF: Minkowski L 1 Metric Probability Distribution Function Area Metric Distance Probability Distribution Function

27 Validation Objectives (Figure of Merits) FOM Mass Flow Rate Temperature Cold Leg Temperature Hot Leg Code RELAP-7 (2014)

28 The uncertainties on the figure of merits are connected to the type of measurements and not specific to a particular detector and location Pressure Uncertainties on Readings ± 0.1 MPa (Primary Pressure) Power ± 1 kw (Core Power) Mass Flow ± kg/s (Mass Flow Rate Primary) Fluid Temperature ± 2 K (Temperature Hot and Cold Leg)

29 Binning of the Data Generated Number of Samples: 8300 Optimized number of bins: 15 Bin Midpoint Bin Count C O U N T S

30 Minkowski L1 Metric CDF relap 0.7 experiment 0.6 CDF Hot Leg Temperature A = K Lower Value Better agreement

31 Probability Distribution Function Area Metric PDF 0.4 PDF relap experiment I = = 27% Higher Percentage Higher Agreement Hot Leg Temperature

32 Distance Probability Distribution Function PDF Fz µ (d) = σ (d) = z [kg/s]

33 Use of Voronoi Tessellation RAVEN posses several sampling strategy Different sampling strategy generate different filling of the input space and different weight-point association The Voronoi tessellation is a common statistical representation Sampler Sampler Sampler Voronoi representation Grid Statistical post processing LHS 33

34 Voronoi tessellation Response Space Tessellation of the Response Space Y Voronoi Tessellation Y X For each point a weight is computed in the probability space These weights are used to construct the variate distribution in the response space and, if applied on the input space, can be used for the computation of the statistical moments without ad-hoc strategies dependent on the sampling methodology X

35 Voronoi tessellation advantages The methodology has been applied to generalize the validation methodologies previously implemented in RAVEN, demonstrating its validity The generality of the approach provides the following advantages: The validation metrics are not dependent on the employed sampling strategy If the reliability weights are computed in the CDF space, there is no need to create ad-hoc weight generation strategies for the future sampling methods The approach can be used for the approximated computation of joint probability functions (crucial for the computation of correlation and covariance matrices when the targeted variables are differently weighted)

36 Conclusions Distribution modeling and sampling strategies have a good degree of maturity Static comparison between model output and experimental distribution is feasible but in an early stage Developments are needed in: Time dependent Correlation in input/output in experimental data needs to be accounted for Extrapolation is a new field which hopefully will see growing capabilities in the next years

37 Thank you Questions? 37

38 Sustaining National Nuclear Assets 38