Neutron inverse kinetics via Gaussian Processes

Transcription

1 Neutron inverse kinetics via Gaussian Processes P. Picca Politecnico di Torino, Torino, Italy R. Furfaro University of Arizona, Tucson, Arizona

2 Outline Introduction Review of inverse kinetics techniques Analytical methods Neural network approach Gaussian Process principles Inverse kinetics via GPs Results and conclusions 2

3 Introduction Accelerator-Driven System (ADS): Nuclear waste transmutation Subcritical systems, steady-state neutron population sustained through an external source (fusion or spallation nuclear reactions) Assessment of subcriticality level Different from critical system (well-defined physical state) Uncertainties and model approximations can deeply affect the accuracy of eigenvalue calculation 3

4 Monitoring of subcriticality level Reactivity evaluation through measurements Characterization of ADS via kinetics features k eff = Interpretation via a model-based inversion 4

5 Review of classical techniques The inversion is often performed on point kinetic equations, which in specific cases allows an analytical evaluation of reactivity Original work by Sjöstrand (1956) Improvements: Gozani (1962) Garelis and Russell (1963) Ravetto et al. ( ) Limitations concerning the capability of dealing with other kinetic models 5

6 Experiments in ADS Typical ADS experiment set-up Compact core design Decoupled systems with strong heterogeneity Localized source Strong space-energy transients Inaccuracy of PK model Need of inversion techniques that can handle more sophisticated kinetic models 6

7 Neural-based inversion Artificial neural networks (ANN) are computational tools that can learn patterns and/or functional relationships in data The basic idea is that the ANN can be optimized to fit a data in the training set Supervised learning using a backpropagation ANN can invert any kinetic model Inverse spatial kinetics (Picca et al., 2009) Inverse MPK (Picca et al., 2010) 7

8 Criticisms to ANN Mathematical background for the interpretation of the ANN results Black-box tool? Design of the ANN architecture what architecture? what activation functions? what learning rate? etc. Reliability of the results Error control? Some criticisms come from a limited understanding of the technique => Examples of extensive use in other domains 8

9 Gaussian Processes principles GPs are widely known in kernel machines area of machine learning Applications in meteorology and geology Gaussian Processes provide a principled, practical, probabilistic approach to learning in kernel machines References: online community ( Rasmussen and Williams, Gaussian Processes for Machine Learning, the MIT Press, 2006 => used in the following as reference for theory (R&W, 2006) 9

10 Properties of GPs GP is a generalization of a finite-dimensional Gaussian-distributed random variable to functions Mathematically equivalent to many well known models Bayesian linear models, spline models, support vector machines, etc. NN tends to GP in the limit of infinite size (Neal, 1996) GP are easier to interpret than their neuralbased counterparts Bayesian inference framework 10

11 Plan of the GP tutorial 1. Regression principles 2. Bayesian regression Difference with respect to classical regression 3. Gaussian processes Advantage compared to general BR 4. Model selection 5. Cross-validation 11

12 Regression principles Regression consists in modeling the relationship between an input and an output variable Various possibility: 1. Linear regression 2. Polynomial regression Coefficients (or weights) can typically be determined using, for instance, the least square method (LS) 12

13 Bayesian regression (I) In 2004 s Technological Review (MIT), Bayesian machine learning were considered among the 10 Emerging Technologies That Will Change the World An estimate of the probability of weights given the data, i.e., can be performed through the Bayes rule (1763): prob. density of the observations, given the parameters beliefs about the prior param. before observations normalizing condition 13

14 Bayesian regression (I) In 2004 s Technological Review (MIT), Bayesian machine learning were considered among the 10 Emerging Technologies That Will Change the World An estimate of the probability of weights given the data, i.e., can be performed through the Bayes rule (1763): prob. density of the observations, given the parameters beliefs about the prior param. before observations normalizing condition 14

15 Bayesian regression (II) Difference between Bayesian and non- Bayesian regression: Non-Bayesian: a single set of parameters are chosen by some criterion (e.g. least square) Bayesian: averaging the output of all possible linear models w.r.t. the Gaussian posterior 15

16 Bayesian regression (III) In Bayesian inference, the evaluation of the marginal likelihood integrals may be difficult The choice of a GP for Bayesian inference allows the integral to be analytically tractable Example with linear regression: likelihood posterior prediction All pdf are Gaussian!! 16

17 GPs for Bayesian regression Gaussian processes (GP) can be interpreted as a linear regression model with an infinite basis functions number (R&W, 2006) Kernel function: Covariance (or kernel) function: 17

18 Kernel function properties Interpretation of covariance (or kernel) function: with It can be shown that are the eigenfunctions of the kernel, It is often less computational intensive to compute the kernel function than the feature vectors itselves ( kernel trick ) A variety of possible choices for covariance function, which leads to different regression models 18

19 Example of kernel function An option is the squared exponential (SE) covariance function It corresponds to a Bayesian linear regression model with infinite Gaussian-shaped basis functions It is defined through hyperparameters lenght scale signal variance { noise variance 19

20 Model selection (I) Model choice for GPs consists in: 1. Selection of the covariance function 2. Definition of the hyperparameters In many practical applications, not easy to specify the covariance function with confidence Principle of the Bayesian inference: the prior, which encodes the a priori knowledge, is updated through the likelihood to obtain a refined understanding of the problem (posterior) 20

21 Model selection (II) The log marginal likelihood can be written as: Gram matrix inversion ~ O(n 3 ) Gram matrix: K i,j =k(x i,x j ) Estimation of by maximizing the marginal function (Rasmussen and Williams, 2006) Computation of derivatives ~ O(n 2 ) 21

22 Cross validation After training, GP is optimized to represent the data in the training set How about cases not in the training set? Cross validation consists in testing the GP prediction capabilities Compute GP output on unseen test examples Analog of the generalization phase in ANN Various estimator can be computed during training to assess GP performance E.g. leave-one-out estimator 22

23 Result summary Inverse kinetics using energy release o Single-input Inverse kinetics using detailed power evolution o Multi-input Implementation detail: Squared exponential kernel functions Single output GP (subcriticality level) GP code source: 23

24 Application of GPs to inverse kinetics Basic test considering a single input and single output problem Given the energy release in a source pulse problem, compute the subcriticality level 24

25 Results (I) Behavior of reactivity as a function of the energy release in a pulsed transient % confidence k eff 0.7 k eff E(0;T) x 10-4 training on 5 examples E(0;T) x 10-4 training on 10 examples 25

26 Results (II) Behavior of reactivity as a function of the energy release in a pulsed transient k eff Training on 50 cases E(0;T) x

27 Cross-validation training on 10 examples training on 50 examples difference between keff and GP prediction standard deviation evaluated by GP (68% confidence) 27

28 Another test case Interpretation of the local power measurement along the transient Multi-input single-output GP 28

29 Results Training on 50 cases Integral energy release Detailed power evolution (10 time measurements) 29

30 Conclusions A novel method for the interpretation of pulsed transients in ADS is proposed The techniques, based on GPs, provides an interesting tool for inverse kinetics Flexible Computationally performing Results are very interesting as the deviation between GP prediction and the actual subcriticality can be theoretically estimated Future works will apply GP to more realistic configurations 30

31 Neutron inverse kinetics via Gaussian Processes P. Picca, R. Furfaro

32 Application of GPs Applications of GP as function approximation: 1. Classification : assign an input pattern to a class (e.g. handwritten digital recognition) 2. Regression : prediction of a continuous function (see next slides) Currently, widespread use only in specific areas E.g. meteorology, geology 32

33 Discussion GP gives a robust statistical ground (i.e. Bayesian inference) to inversion problem Marginal likelihood properties allows GPs to automatically incorporate a trade-off between model fit and model complexity (R&W, 2006) Marginal likelihood still tend to favour the least complex model able to explain the data Design phase become less crucial (vs ANN) ANN provides information about what the useful features are for solving a particular problem E.g. part of transient more meaningful 33

34 Hierarchical Bayesian inference Bayesian inference various level: I. Level 1: parameter level likelihood posterior prior II. Level 2: hyper-parameter level III. Level 3: model structures 34

35 Historical perspective Traditionally, fixed basis functions were used for classical regression Advantage of ANN in the adaptive basis functions more flexibility Practical problems: results may depends on ANN design how to ensure ANN reliability? Limitation of fixed basis functions were overcome through the advancement in kernel machine Gaussian Processes equivalent to an infinite set of basis function (R&W, 2006) 35