Cross-Validation in the Cluster Expansion

Transcription

1 Cross-Validation in the Cluster Expansion Axel Hübner Berlin, September 27, 2017 Solid State Theory Group Institut für Physik / Humboldt-Universität zu Berlin

2 Problem Thermoelectric alloy Ba 8 Al x Si 46 x M. Troppenz, S. Rigamonti and C. Draxl, Chem. Mater (2017) 1

3 Cluster expansion Non-substituted crystal A. van d. Walle ; G. Ceder Journal of Phase Equilibria 23 (2002), Nr. 4 2

4 Cluster expansion Non-substituted crystal Configuration σ A. van d. Walle ; G. Ceder Journal of Phase Equilibria 23 (2002), Nr. 4 3

5 Cluster expansion Non-substituted crystal Configuration σ Clusters α Correlation between σ and α X σα A. van d. Walle ; G. Ceder Journal of Phase Equilibria 23 (2002), Nr. 4 4

6 Cluster expansion Non-substituted crystal Configuration σ Clusters α Correlation between σ and α X σα Model Ê σ = α X σα J α A. van d. Walle ; G. Ceder Journal of Phase Equilibria 23 (2002), Nr. 4 5

7 Mean squared error Mean squared error MSE = 1 n n σ=1 [ ] 2 E σ Êσ(J) L. Fahrmeir, T. Kneib and S. Lang: Regression - Modelle, Methoden und Anwendungen, Springer-Verlag Berlin Heidelberg, p

8 Mean squared error Mean squared error MSE = 1 n n σ=1 [ ] 2 E σ Êσ(J) J = (X X ) 1 X E Ê = X J = PE L. Fahrmeir, T. Kneib and S. Lang: Regression - Modelle, Methoden und Anwendungen, Springer-Verlag Berlin Heidelberg, p

9 Mean squared error Mean squared error MSE = 1 n n σ=1 [ ] 2 E σ Êσ(J) J = (X X ) 1 X E Ê = X J = PE Projector P = X (X X ) 1 X L. Fahrmeir, T. Kneib and S. Lang: Regression - Modelle, Methoden und Anwendungen, Springer-Verlag Berlin Heidelberg, p

10 Cross-validation Cross-validation score CV 2 = 1 n n [ E σ Ê σ (σ) σ=1 σ: sample (configuration) n: training set size E σ : ab-initio energy for sample σ Ê (σ) σ : predicted energy excluding sample σ ] 2 7

11 Example: Cross-validation E(σ) = 0.5(σ 0.5) ε Ê(σ) = α X σα J α 8

12 Example: Cross-validation m X σα = σ α (α = 0...m) Ê(σ) = σ α J α α=0 9

13 Example: Cross-validation The CV detects overfitting: optimal model can be found 10

14 Performance 11

15 Cross-validation Leave-one-out CV CV 2 = 1 n ( n E σ Êσ 1 p σσ σ=1 ) 2 A. van d.walle, G. Ceder Journal of Phase Equilibria 23 (2002), Aug., Nr. 4 12

16 Cross-validation Leave-one-out CV CV 2 = 1 n ( n E σ Êσ 1 p σσ σ=1 ) 2 A. van d.walle, G. Ceder Journal of Phase Equilibria 23 (2002), Aug., Nr. 4 In this work Leave-many-out cross-validation 12

17 Cross-validation Leave-one-out CV CV 2 = 1 n ( n E σ Êσ 1 p σσ σ=1 ) 2 A. van d.walle, G. Ceder Journal of Phase Equilibria 23 (2002), Aug., Nr. 4 In this work Leave-many-out cross-validation Instability 12

18 Cross-validation Leave-one-out CV CV 2 = 1 n ( n E σ Êσ 1 p σσ σ=1 ) 2 A. van d.walle, G. Ceder Journal of Phase Equilibria 23 (2002), Aug., Nr. 4 In this work Leave-many-out cross-validation Instability Relation to noise 12

19 Results: LMOCV MSE for not excluded configurations J (E) Leave-many-out cross-validation score CV 2 = 1 N N j=1 1 ℵ(E j ) ( ) ) 1 2 I P Ej (ÊEj E Ej 2 13

20 Results: LMOCV MSE for not excluded configurations J (E) Leave-many-out cross-validation score CV 2 = 1 N N j=1 1 ℵ(E j ) ( ) ) 1 2 I P Ej (ÊEj E Ej 2 What is faster? Old or new method? 13

21 Performance LMOCV for a clathrate alloy. n = 1232, N = 1, m = 80. Energies found with effective-medium-theory calculator. 14

22 Performance LMOCV for a clathrate alloy. n = 1232, N = 1, ℵ =

23 Configuration selection CV 2 = 1 n n σ=1 ( E σ Êσ 1 p σσ ) 2 when pnn 1 16

24 CV-noise relation Limit number of ab-initio calculations Missing clusters error comparable to noise (1) Use Fischer-Courant s theorem Relation For the model with the lowest CV and (1 ) holds: E MSE CV λ A A max λ A A max is the maximum eigenvalue of A A, (A) ij = δ ij p ij 1 p ii. 17

25 CV-noise relation 100 random models per noise level 18

26 A real alloy Find optimal model for clathrate alloy Ba 8 Al x Si 46 x. n = 107 Data from M. Troppenz, S. Rigamonti and C. Draxl, Chem. Mater (2017) 19

27 Conclusions Expression for LMOCV Origin of instabilities in formula Devised ways to circumvent instabilities Found CV-noise relation Criterion to limit computational effort 20

28 Our team Figure 1: Claudia Draxl Figure 2: Santiago Rigamonti 21

29 Thanks for your attention! 21