Machine learning for crystal structure prediction

Size: px

Start display at page:

Download "Machine learning for crystal structure prediction"

Brenda Black
6 years ago
Views:

org Computational Material Design Laboratory

1 Machine learning for crystal structure prediction Fei Qi Computational Material Design Laboratory Moscow Institute of Physics and Technology Dolgoprudny, Moscow December 12, 2014

2 Motivation Introduction to Machine Learning Machine Learning based Property Prediction Results of KRR based Property Prediction Discussions and Conclusions

3 Complexity of Crystal Structure Prediction Crystal structure prediction (CSP) To find stable crystal structures To find a structure with some expected physical/chemical properties. Computation of properties First principle calculation (FPC) is with very high complexity. In CSP, a large number of structures are populated and evaluated using FPC The most time consuming step in CSP.

4 Evolutionary Crystal Structure Prediction USPEX 1 provides a smart evolutionary approach to populate structures. But the number of structures to be evaluated is still large. Workflow of USPEX 1. Initial structure population 2. Applying evolutionary operators to structures 3. Structure relaxation and property computation 4. New generation 1 C. W. Glass, A. R. Oganov, and N. Hansen, Computer Physics Communications 175, 713 (2006).

5 Evolutionary Crystal Structure Prediction USPEX 1 provides a smart evolutionary approach to populate structures. But the number of structures to be evaluated is still large. Workflow of USPEX 1. Initial structure population 2. Applying evolutionary operators to structures 3. Structure relaxation and property computation 4. New generation Can we speed-up the process? 1 C. W. Glass, A. R. Oganov, and N. Hansen, Computer Physics Communications 175, 713 (2006).

6 Evolutionary Crystal Structure Prediction USPEX 1 provides a smart evolutionary approach to populate structures. But the number of structures to be evaluated is still large. Workflow of USPEX 1. Initial structure population 2. Applying evolutionary operators to structures 3. Structure relaxation and property computation 4. New generation Can we speed-up the process? To provide an efficient alternative to step 3. Or to filter structures after step 2. 1 C. W. Glass, A. R. Oganov, and N. Hansen, Computer Physics Communications 175, 713 (2006).

7 Introduction to Machine Learning Data Algorithm Model f (x)

8 Supervised Learning for Property Prediction Data Algorithm Model f (x)

9 Supervised Learning for Property Prediction Data Algorithm Model f ( ) =

10 y To Speed-Up Property Computation with Machine Learning Machine learning (ML) can learn models from data. Training is to build the model according to known data. Prediction applies model on unknown data. ML can accelerate property computation The training procedure is very efficient (seconds vs hours) The prediction procedure is extremely fast (ms vs hours) A typical ML approach is regression analysis Model y =2x +1 Linear Regression Kernel Ridge Regression Model y =sin(x +π/7) +1 Kernel Regression y =2x x x

11 Applying machine learning applied to CSP Property Prediction Model P = f (X) Feature Extraction X

12 Feature extraction Feature vector Coordinates of atoms Bonding length Bonding angles...

13 Feature extraction Feature vector Coordinates of atoms Bonding length Bonding angles... Challenges in CSP Performance depends on feature vector Need to be designed for different compounds Leads to complex models Returns back to FPC Difficult to train

14 Feature extraction: an exemplar-based approach Exemplar feature vector Similarities to reference structures K(, ) K(, ) K(, ).

etc. Energy/property learning shares some properties with image processing High-dimensional space Globally non-linear but locally linear

15 Motivation Formulation Method Results Summary Why examples makes sense Intuitively, similar structure are with similar property. Exemplar-based approaches have successful applications in the image processing community, including image inpainting2, super-resolution, etc. Energy/property learning shares some properties with image processing High-dimensional space Globally non-linear but locally linear property. 2 C. Guillemot and O. Le Meur, Image Inpainting: Overview and Recent Advances, IEEE Signal Processing Magazine, 31(1): , Jan

16 The Kernel Ridge Regression Model The property of an unknown structure is predicted using P pred (f) = N i=1 α ik (f i, f), where f and f i are fingerprints of the unknown and known structures. The training minimize the following goal function N i=1 Pi prediction Pi train 2 + λ N The optimization has a closed form solution α = (K + λi) 1 P train, where the kernel matrix K is K ij = K ( f i, f j ). i=1 α2 i

17 The Fingerprint Function The Gaussian kernel is used, which is ( / K(x, t) exp x t 2 2 2σ 2). The fingerprint provides a consistent representation is V uc δ(r R ij) f AB (R) = 4π N A N B Rij 2 1. The corresponding descriptor is obtained by discretization f AB = [f AB (0), f AB ( ), f AB (2 ),, f AB (n )] T, where A and B can be all combinations of A, B fulfill A B. A i B j

18 Sparsity and feature/sample selection Sparsity If we have a large population of referencing structures, only a small portion of them are useful. Vector form of the KRR model Kα P train λ α 2 2 Applying sparsity into the model Kα P train λ α 0 Difference between atoms can be considered. Using atomic fingerprint Number of training samples is less than that of atoms. Assumes non-uniform contributions to a property

19 Evaluation Method Performance measurement Person correlation coefficient K-fold cross-validation 1 2 K Divide the training set to K subsets To select 1 subset for testing Other K 1 subsets are used to train the model

20 Evaluation Method Performance measurement Person correlation coefficient K-fold cross-validation 1 2 K Divide the training set to K subsets To select 1 subset for testing Other K 1 subsets are used to train the model To guarantee for generalization capability

21 Prediction results of properties of BN Predicted properties of BN: density (left), hardness (upper right), and bandgap (bottom right).

22 Prediction results of average enthalpy Prediction results of average enthalpy of: left) BN with variable combination and right) Ag2Se with 24 atoms at 30GPa.

23 Predicting atomization energy of organic molecules The QM7 dataset contains 7165 molecules The median absolute error is 7.6 kcal/mol Error of reference 4 is 7.6 kcal/mol. Results on the QM7 dataset M. Rupp, A. Tkatchenko, K.-R. Mller, and O. A. von Lilienfeld, Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning, Physics Review Letters, 108(5), Jan

24 To boost the prediction with USPEX Pearson Coefficients 100% 95% 90% 85% 80% 75% ML accuracy w.r.t. generation on average enthalpy of BN Density Bandgap Hardness Enthalpy_avg 70% 0% 20% 40% 60% 80% 100% Percentage of training samples Training samples are increased according to USPEX s evolutionary generations. The rest samples are used in testing. The generalization capability increases along with the generation.

25 Summary Conclusions Exemplar-based feature vector is introduced for CSP with machine learning. Kernel ridge regression (KRR) can be used in the prediction of material properties. Sparsity and feature selection are briefly introduced. With machine learning, structures (populated by USPEX) can be pre-filtered (or evaluated very efficiently with tolerable errors).

26 Thanks for your attention. Acknowledgments Artem R. Oganov (State University, New York & MIPT) Qiang Zhu (State University, New York) Guangrui Qian (State University, New York) Qingfeng Zeng (Northwestern Polytechnique University, China) Haiyang Niu, Qinggao Wang and Xiaohu Yu (MIPT) This work is funded by the grant of the Government of the Russian Federation NO. 14.A

Supervised Machine Learning: Learning SVMs and Deep Learning. Klaus-Robert Müller!!et al.!!

Supervised Machine Learning: Learning SVMs and Deep Learning Klaus-Robert Müller!!et al.!! Today s Tutorial Machine Learning introduction: ingredients for ML Kernel Methods and Deep networks with explaining