Pseudo-measurement simulations and bootstrap for the experimental cross-section covariances estimation with quality quantification

Size: px

Start display at page:

Download "Pseudo-measurement simulations and bootstrap for the experimental cross-section covariances estimation with quality quantification"

Sharleen Carr
5 years ago
Views:

1 Pseudo-measurement simulations and bootstrap for the experimental cross-section covariances estimation with quality quantification S. Varet 1 P. Dossantos-Uzarralde 1 N. Vayatis 2 E. Bauge 1 1 CEA-DAM-DIF 2 ENS Cachan WONDER-2012 September S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

2 Motivations: experimental cross-sections n2n cross section Mn n2n cross section 55 EXFOR data Available informations: the measurements their uncertainty Energy (MeV) covariance matrix estimation and its inverse Example Evaluated cross sections uncertainty: generalized χ 2 ([VDUVB11]) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

3 Motivations: experimental cross-sections covariances HOW? Empirical estimator: only one measurement per energy cov(f i, F j ) 1 n n k=1 (F (k) i F i )(F (k) j F j ) Conventionnal approach: propagation error formula [IBC04],[Kes08] S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

4 Motivations: experimental cross-sections covariances HOW? Empirical estimator: only one measurement per energy Conventionnal approach: propagation error formula [IBC04],[Kes08] cov(f i, F j ) param. exp. (q k, q l ) F i q k. F j q l.cov(q k, q l ) ex. : fission fragments yield, recoil protons yield, the needed informations are rarely available linearity assumption S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

5 Motivations: experimental cross-sections covariances HOW? Empirical estimator: only one measurement per energy Conventionnal approach: propagation error formula [IBC04],[Kes08] QUALITY OF THE COVARIANCES ESTIMATION? S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

6 Outline 1 Notations 2 Experimental covariances estimation: new method 3 Validation 4 Quality measure of the obtained estimation 5 Conclusion S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

7 Outline Notations 1 Notations 2 Experimental covariances estimation: new method 3 Validation 4 Quality measure of the obtained estimation 5 Conclusion S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

8 Notations Notations Schematic representation of the experimental cross sections S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

9 Outline Experimental covariances estimation: new method 1 Notations 2 Experimental covariances estimation: new method 3 Validation 4 Quality measure of the obtained estimation 5 Conclusion S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

10 Experimental covariances estimation: new method Pseudo-measurements method Given the available information (the measurements and their uncertainty) The idea: Repeat the available information in order to artificially increase the number of measurements The method: 1 Construction of a regression model h (SVM, polynomial,...) 2 Generation of r pseudo-measurements (S (1),..., S (r) ): gaussian noise centered on h: N ((h(e 1 ),..., h(e N )) t, diag(σ 1,..., σ N )) 3 ΣF : empirical estimator of Σ F 4 Diagonal terms are imposed S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

11 Experimental covariances estimation: new method Pseudo-measurements method Given the available information (the measurements and their uncertainty) The idea: Repeat the available information in order to artificially increase the number of measurements The method: 1 Construction of a regression model h (SVM, polynomial,...) 2 Generation of r pseudo-measurements (S (1),..., S (r) ): gaussian noise centered on h: N ((h(e 1 ),..., h(e N )) t, diag(σ 1,..., σ N )) 3 ΣF : empirical estimator of Σ F 4 Diagonal terms are imposed S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

12 Experimental covariances estimation: new method The SVM regression principle [SS04], [VGS97] Linear case: Assume F i = h(e i ) = w.e i + b with E i R s and w R s (here s = 1). The svm regression model is a solution of the constraint optimisation problem: flat function: minimize errors lower than ε: subject to 1 2 w 2 { Fi (w.e i + b) ε (w.e i + b) F i ε Non linear case: Find a map of the initial space of energy, (into a higher dimensionnal space) such as the problem becomes linear in the new space (mapping via Kernel). S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

13 Experimental covariances estimation: new method Pseudo-measurements method Given the available information (the measurements and their uncertainty) The idea: Repeat the available information in order to artificially increase the number of measurements The method: 1 Construction of a regression model h (SVM, polynomial,...) 2 Generation of r pseudo-measurements (S (1),..., S (r) ): gaussian noise centered on h: N ((h(e 1 ),..., h(e N )) t, diag(σ 1,..., σ N )) 3 ΣF : empirical estimator of Σ F 4 Diagonal terms are imposed S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

14 Experimental covariances estimation: new method Pseudo-measurements method Σ F term at the i th line and j th column, for i and j {1,..., N}: and C ij = σ iσ j n + r n k=1 + σ iσ j n + r (F (k) i r k=1 (S (k) i h(e i ))(F (k) j h(e j )) V i V j h(e i ))(S (k) j h(e j )) V i V j where n = 1. C ii = σ 2 i S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

15 Experimental covariances estimation: new method Number r of pseudo-measurements? 0 Constraint on r r too small: Σ F non invertible r too large: Σ F diagonal matrix r = 0 Σ F invertible no pseudo-measurements ΣF non invertible r such as Σ F invertible S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

16 Outline Validation 1 Notations 2 Experimental covariances estimation: new method 3 Validation 4 Quality measure of the obtained estimation 5 Conclusion S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

17 Toy model Validation Goal: Compare Σ F with Σ F : real case: Σ F is unknown we build a toy model with Σ F fixed numerical validation M := M 1. M N Toy Model: ( F) N N (m M, Σ M ) number of M realisations n = 1, N = θ = B A ΣF = B A S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

18 Validation Pseudo-measurements with the Toy Model 10 SVM regression on the toy model 5 Measurements (ex : Xs) m M Variable (ex : energy) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

19 Validation Pseudo-measurements with the Toy Model 10 SVM regression on the toy model 5 Measurement (ex : Xs) m M Toy measurement Variable (ex : energy) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

20 Validation Pseudo-measurements with the Toy Model 10 SVM regression on the toy model 5 Measurement (ex : Xs) SVM regression m M Toy measurement Variable (ex : energy) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

21 Validation Pseudo-measurements with the Toy Model 10 SVM regression on the toy model 5 Measurements (ex : Xs) SVM regression m M Toy measurement r=1 pseudo measurement Variable (ex : energy) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

22 Validation Pseudo-measurements with the Toy Model N=5, r = 1 sample of pseudo-measurements Real matrix Estimation S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

23 Validation 55 25Mn: (n,2n) cross section Values extracted from [MTN11] N=11, r=1, Σ F is a 11x11 matrix SVM model Experimental cross sections Pseudo measurements Cross section (mb) Energy (ev) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

24 Validation 55 25Mn: total cross section Values extracted from EXFOR [EXF] N=399, r=1, Σ F is a 399x399 matrix 4 SVM model Experimental data Pseudo measurements Cross section (mb) Energy (ev) x How far is the estimation from the real matrix? S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

25 Outline Quality measure of the obtained estimation 1 Notations 2 Experimental covariances estimation: new method 3 Validation 4 Quality measure of the obtained estimation 5 Conclusion S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

26 Quality measure of the obtained estimation Estimator quality Criterion choice: quality of Σ F and/or quality of Σ F 1? distance criterion 1 Distance between Σ F and Σ F : Frobenius norm 1 2 N N IE( Σ F Σ F fro ) = IE Ĉ ij C ij 2 i=1 j=1 2 Distance between Σ 1 F and Σ 1 F : Kullback-Leibler distance [LRZ08] KL( Σ F, Σ F ) = trace(( Σ F ) 1 1.Σ F ) log(det( Σ F.ΣF )) N Criterion estimation parametric bootstrap S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

27 Quality measure of the obtained estimation Bootstrap principle F : θ F, Σ F : unknown (F (1),..., F (n) ) : h, Σ F : sample measurements (F (1),..., F (n) ) 1,..., (F (1),..., F (n) ) B resample measurements Σ F 1 Σ F B Resampling allows to make statistics on Σ F : ÎE( Σ F ), Var( Σ F ), Bias( Σ F ),... Resample strategy: classical: draw with replacement in the initial sample (F (1),..., F (n) ) parametric: simulations with a fixed law (here N (h, Σ F )) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

28 Quality measure of the obtained estimation Estimation of IE( Σ F Σ F fro ) and KL( Σ F, Σ F ) 1 st case: with r = 0, Σ F invertible Usual parametric bootstrap: (F (1),..., F (n) ) 1,..., (F (1),..., F (n) ) B where F (i) N (H, Σ F ) with H = (h(e 1 ),..., h(e N )) t. cσ F b term at the i th line and j th column, for i and j {1,..., N} and for b = 1,..., B: C b ij = σiσj n nx k=1 (F (k)b i h(e i))(f (k)b j h(e j)) q q bv bv i j and C b ii = σ 2 i IE( Σ F Σ F ) 1 B KL( Σ F, Σ F ) 1 B B b ΣF Σ F b=1 B KL( ΣF b, Σ F ) b=1 S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

29 Quality measure of the obtained estimation Estimation of IE( Σ F Σ F fro ) and KL( Σ F, Σ F ) 2 nd case: with r = 0, Σ F non invertible 2 simultaneous parametric bootstraps: (F (1),..., F (n) ) 1,..., (F (1),..., F (n) ) B where F (i) N (H, c Σ F ) et (S (1),..., S (r) ) 1,..., (S (1),..., S (r) ) B where S (i) N (H, diag( c Σ F )) cσ F b term at the i th line and j th column, for i and j {1,..., N} and for b = 1,..., B: C b ij = σiσj n + r nx k=1 (F (k)b i h(e i))(f (k)b j h(e j)) q q bv bv i j + σiσj n + r rx k=1 (S (k)b i h(e i))(s (k)b j h(e j)) q q bv bv i j and C b ii = σ 2 i IE( Σ F Σ F ) 1 B KL( Σ F, Σ F ) 1 B B b ΣF Σ F b=1 B KL( ΣF b, Σ F ) b=1 S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

30 Quality measure of the obtained estimation Bootstrap on the Toy Model N = 5, n = 1, B = 30, Σ F = , r = 1, Empirical mean and variance of C2 F Σ F its bootstrap estimation Empirical mean and variance of C2 F Σ F its bootstrap estimation (B=30). C2 F Σ F Σ B b=1 C2 b F C2 F /B Index of the 1 sample of measures S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

31 Quality measure of the obtained estimation Bootstrap on the Toy Model N = 5, n = 1, B = 30, Σ F = , r = 1, Empirical mean and variance of C2 F Σ F its bootstrap estimation Empirical mean and variance of C2 F Σ F its bootstrap estimation (B=30). C2 F Σ F Σ B b=1 C2 b F C2 F /B Index of the 1 sample of measures Empirical mean and variance of KL and its bootstrap estimation Empirical mean and variance of KL and its bootstrap estimation (B=30) Index of the 1 sample of measures KL KL boot S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

32 Quality measure of the obtained estimation 55 25Mn: (n,2n) cross section [MTN11] SVM model Experimental cross sections Pseudo measurements Cross section (mb) Energy (ev) IE( Σ F Σ F ) IE(KL( Σ F )) Var( Σ F Σ F ) Var(KL( Σ F )) S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

Quality measure of the obtained estimation 55 25Mn: total cross section 4 SVM model Experimental data Pseudo measurements 50 0.025 Cross section (mb) 3.5 3 100 150 200 250 0.02 0.015 0.01 0.

33 Quality measure of the obtained estimation 55 25Mn: total cross section 4 SVM model Experimental data Pseudo measurements Cross section (mb) Energy (ev) x IE( Σ F Σ F ) IE(KL( Σ F )) Var( Σ F Σ F ) Var(KL( Σ F )) 6.19 S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

34 Outline Conclusion 1 Notations 2 Experimental covariances estimation: new method 3 Validation 4 Quality measure of the obtained estimation 5 Conclusion S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

35 Conclusion Conclusion Σ F estimation 1 Classical approaches one measurement per energy experimental informations rarely available quite unrealistic assumptions 2 Our approach: only measurements are needed quality measure of the estimation Estimator quality Numerical validation Application S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

36 Conclusion Conclusion Σ F estimation Estimator quality Estimation of b Σ F Σ F with Bootstrap Estimation of KL( b Σ F) with Bootstrap Numerical validation Application S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

37 Conclusion Conclusion Σ F estimation Estimator quality Numerical validation Toy model Application S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

38 Conclusion Conclusion Σ F estimation Estimator quality Numerical validation Application Application to 55 25Mn S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

39 Future work Conclusion New approach Optimisation: shrinkage approach (in progress) Other approaches Kriging: cf poster session S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

40 References Conclusion Experimental Nuclear Reaction Data EXFOR, M. Ionescu-Bujor and D. G. Cacuci, A comparative review of sensitivity and uncertainty analysis of large-scale systems i: Deterministic methods, Nuclear Science and Engineering 147 (2004), Grégoire Kessedjian, Mesures de sections efficaces d actinides mineurs d intérêt pour la transmutation, Ph.D. thesis, Bordeaux 1, E. Levina, A. Rothman, and J. Zhu, Sparse estimation of large covariance matrices via a nested lasso penalty, The Annals of Applied Statistics 2 (2008), no. 1, A. Milocco, A. Trkov, and R. Capote Noy, Nuclear data evaluation of 55 mn by the empire code with emphasis on the capture cross-section, Nuclear Engineering and Design 241 (2011), no. 4, A.J. Smola and B. Schölkopf, A tutorial on support vector regression, Statistics and Computing 14 (2004), S. Varet, P. Dossantos-Uzarralde, N. Vayatis, and E. Bauge, Experimental covariances contribution to cross-section uncertainty determination, NCSC2 Vienna, V. Vapnik, S. Golowich, and A. Smola, Support vector method for function approximation, regression estimation, and signal processing, Advances in Neural Information Processing Systems 9 (1997), S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER / 29

Kriging approach for the experimental cross-section covariances estimation

Kriging approach for the experimental cross-section covariances estimation EPJ Web of Conferences, () DOI:./ epjconf/ C Owned by te autors, publised by EDP Sciences, Kriging approac for te experimental cross-section covariances estimation S. Varet,a, P. Dossantos-Uzarralde, N.