ELSI: A Unified Software Interface for Kohn-Sham Electronic Structure Solvers

Similar documents
Recent Developments in the ELSI Infrastructure for Large-Scale Electronic Structure Theory

DGDFT: A Massively Parallel Method for Large Scale Density Functional Theory Calculations

A knowledge-based approach to high-performance computing in ab initio simulations.

Comparing the Efficiency of Iterative Eigenvalue Solvers: the Quantum ESPRESSO experience

Verbundprojekt ELPA-AEO. Eigenwert-Löser für Petaflop-Anwendungen Algorithmische Erweiterungen und Optimierungen

Block Iterative Eigensolvers for Sequences of Dense Correlated Eigenvalue Problems

Parallel Eigensolver Performance on the HPCx System

Experiences with Self-Consistent Tight Binding and ELSI

Parallelization of the Molecular Orbital Program MOS-F

CP2K. New Frontiers. ab initio Molecular Dynamics

FEAST eigenvalue algorithm and solver: review and perspectives

Recent developments of Discontinuous Galerkin Density functional theory

Large Scale Electronic Structure Calculations

ESLW_Drivers July 2017

INITIAL INTEGRATION AND EVALUATION

References. Documentation Manuals Tutorials Publications

ELECTRONIC STRUCTURE CALCULATIONS FOR THE SOLID STATE PHYSICS

CRYSTAL in parallel: replicated and distributed (MPP) data. Why parallel?

Making electronic structure methods scale: Large systems and (massively) parallel computing

The ELPA Library Scalable Parallel Eigenvalue Solutions for Electronic Structure Theory and Computational Science

The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and

Before we start: Important setup of your Computer

Parallel Eigensolver Performance on High Performance Computers

PFEAST: A High Performance Sparse Eigenvalue Solver Using Distributed-Memory Linear Solvers

DFT / SIESTA algorithms

Efficient implementation of the overlap operator on multi-gpus

Atomic orbitals of finite range as basis sets. Javier Junquera

Cyclops Tensor Framework

COMPUTATIONAL TOOL. Fig. 4.1 Opening screen of w2web

Massively parallel electronic structure calculations with Python software. Jussi Enkovaara Software Engineering CSC the finnish IT center for science

A Reconfigurable Quantum Computer

ILAS 2017 July 25, 2017

CP2K: Past, Present, Future. Jürg Hutter Department of Chemistry, University of Zurich

Basic introduction of NWChem software

4th year Project demo presentation

DFT in practice. Sergey V. Levchenko. Fritz-Haber-Institut der MPG, Berlin, DE

ab initio Electronic Structure Calculations

Improvements for Implicit Linear Equation Solvers

Parallel Eigensolver Performance on High Performance Computers 1

Opportunities for ELPA to Accelerate the Solution of the Bethe-Salpeter Eigenvalue Problem

Efficient algorithms for symmetric tensor contractions

Large-scale real-space electronic structure calculations

Algorithms and Computational Aspects of DFT Calculations

Matrix Eigensystem Tutorial For Parallel Computation

Mitglied der Helmholtz-Gemeinschaft. Linear algebra tasks in Materials Science: optimization and portability

DFT and beyond: Hands-on Tutorial Workshop Tutorial 1: Basics of Electronic Structure Theory

The Augmented Spherical Wave Method

OPENATOM for GW calculations

Improving the performance of applied science numerical simulations: an application to Density Functional Theory

TR A Comparison of the Performance of SaP::GPU and Intel s Math Kernel Library (MKL) for Solving Dense Banded Linear Systems

Conquest order N ab initio Electronic Structure simulation code for quantum mechanical modelling in large scale

Presentation of XLIFE++

Metal Conquest. Lianheng Tong. 2 nd March 2011

Performance optimization of WEST and Qbox on Intel Knights Landing

Is there a future for quantum chemistry on supercomputers? Jürg Hutter Physical-Chemistry Institute, University of Zurich

Sakurai-Sugiura algorithm based eigenvalue solver for Siesta. Georg Huhs

Introduction to Benchmark Test for Multi-scale Computational Materials Software

Chapter 3. The (L)APW+lo Method. 3.1 Choosing A Basis Set

Eigenvalue Problems CHAPTER 1 : PRELIMINARIES

An Optimized and Scalable Eigensolver for Sequences of Eigenvalue Problems

Divide and conquer algorithms for large eigenvalue problems Yousef Saad Department of Computer Science and Engineering University of Minnesota

Scalable Hybrid Programming and Performance for SuperLU Sparse Direct Solver

Leveraging sparsity and symmetry in parallel tensor contractions

The Quantum ESPRESSO Software Distribution

Time-dependent density functional theory (TDDFT)

Contents. Preface... xi. Introduction...

Density Functional Theory

A Quantum Chemistry Domain-Specific Language for Heterogeneous Clusters

Basic introduction of NWChem software

Preliminary Results of GRAPES Helmholtz solver using GCR and PETSc tools

2.1 Introduction: The many-body problem

Strassen s Algorithm for Tensor Contraction

Accelerating Linear Algebra on Heterogeneous Architectures of Multicore and GPUs using MAGMA and DPLASMA and StarPU Schedulers

Solving the Inverse Toeplitz Eigenproblem Using ScaLAPACK and MPI *

Porting a sphere optimization program from LAPACK to ScaLAPACK

VASP: running on HPC resources. University of Vienna, Faculty of Physics and Center for Computational Materials Science, Vienna, Austria

1 Overview. 2 Adapting to computing system evolution. 11 th European LS-DYNA Conference 2017, Salzburg, Austria

Domain specific libraries. Material science codes on innovative HPC architectures Anton Kozhevnikov, CSCS December 5, 2016

Arnoldi Methods in SLEPc

Preconditioned Eigenvalue Solvers for electronic structure calculations. Andrew V. Knyazev. Householder Symposium XVI May 26, 2005

Opportunities from Accurate and Efficient Density Functional Theory Calculations for Large Systems

Scalability Programme at ECMWF

All-electron density functional theory on Intel MIC: Elk

MODULE 2: QUANTUM MECHANICS. Practice: Quantum ESPRESSO

Weather Research and Forecasting (WRF) Performance Benchmark and Profiling. July 2012

J S Parker (QUB), Martin Plummer (STFC), H W van der Hart (QUB) Version 1.0, September 29, 2015

Susumu YAMADA 1,3 Toshiyuki IMAMURA 2,3, Masahiko MACHIDA 1,3

Density-Matrix-Based Algorithms for Solving Eingenvalue Problems

PASC17 - Lugano June 27, 2017

H ψ = E ψ. Introduction to Exact Diagonalization. Andreas Läuchli, New states of quantum matter MPI für Physik komplexer Systeme - Dresden

CP2K: the gaussian plane wave (GPW) method

Introduction to numerical computations on the GPU

Improving Dynamical Core Scalability, Accuracy, and Limi:ng Flexibility with the ADER- DT Time Discre:za:on

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)

Symmetric Pivoting in ScaLAPACK Craig Lucas University of Manchester Cray User Group 8 May 2006, Lugano

Introduction to Density Functional Theory with Applications to Graphene Branislav K. Nikolić

Accelerating and Scaling Lanczos Diagonalization with GPGPU

The Plane-wave Pseudopotential Method

Performance Analysis of Lattice QCD Application with APGAS Programming Model

MAGMA MIC 1.0: Linear Algebra Library for Intel Xeon Phi Coprocessors

Transcription:

ELSI: A Unified Software Interface for Kohn-Sham Electronic Structure Solvers Victor Yu and the ELSI team Department of Mechanical Engineering & Materials Science Duke University

Kohn-Sham Density-Functional Theory density DGDFT NWCHEM Hc = εsc L.M. Ghiringhelli et al., Towards a Common Format for Computational Materials Science Data, Psi-k Scientific Highlight of the Month 131, July 2016. wave function 1/13

Kohn-Sham Density-Functional Theory density DGDFT NWCHEM density matrix Hc = εsc L.M. Ghiringhelli et al., Towards a Common Format for Computational Materials Science Data, Psi-k Scientific Highlight of the Month 131, July 2016. wave function 1/13

Kohn-Sham Density-Functional Theory density DGDFT NWCHEM density matrix Hc = εsc L.M. Ghiringhelli et al., Towards a Common Format for Computational Materials Science Data, Psi-k Scientific Highlight of the Month 131, July 2016. 1/13

Kohn-Sham Density-Functional Theory density DGDFT NWCHEM density matrix Hc = εsc L.M. Ghiringhelli et al., Towards a Common Format for Computational Materials Science Data, Psi-k Scientific Highlight of the Month 131, July 2016. wave function 1/13

The Cubic Wall in Kohn-Sham Density-Functional Theory DFT-PBE with FHI-aims Graphene monolayer (2D) From 4,050 to 7,200 atoms 14 numeric atom-centered orbitals per carbon atom The O(N 3 ) Kohn-Sham eigenproblem must be addressed by all electronic structure codes based on KS-DFT O(N) O(N 3 ) Edison (Intel Ivy Bridge) 80 nodes x 24 CPU cores per node 2/13

Kohn-Sham Electronic Structure Solvers ELPA (eigensolution) Dense, direct eigensolver O(N 3 ) libomm (density matrix) Orbital minimization O(N 3 ) reduced prefactor PEXSI (density matrix) Pole expansion, selected inversion O(N 2 ) SIPs (eigensolution) Sparse, iterative eigensolver O(N 2 ) CheSS (density matrix) Chebyshev expansion O(N) And many others, e.g. iterative solvers and ScaLAPACK A unified software interface and solver comparison platform 3/13

ELSI Timeline ELSI Kick-Off Meeting, Duke University, Durham, 2017 Aug 2015

ELSI: Connection between DFT Codes and KS Solvers Create Hamiltonian and overlap matrices Different matrix storage strategies Different application programming interfaces Different programming languages Compute eigenvalues, eigenvectors, density matrix 4/13

ELSI: Connection between DFT Codes and KS Solvers Create Hamiltonian and overlap matrices Convert data format and distribution automatically Analyze the problem: Basis set size, sparsity, desired output, available computational resource, (ongoing) Suggest a solver with reasonable settings (future) Compute eigenvalues, eigenvectors, density matrix 4/13

ELSI Timeline Electronic Structure Library Coding Workshop, ZCAM, Zaragoza, 2016 Aug 2015 Jun 2016

Philosophy of the ELSI Interface Easy to implement Designed to be flexible for rapid integration into a variety of KS-DFT codes Versatile Includes density matrix and eigensystem formalisms on equal footing Flexible All technical settings in solver libraries are adjustable for experienced users API of ELSI interface elsi_init! BEGIN SCF LOOP elsi_set elsi_{dm ev}! END SCF LOOP elsi_finalize 5/13

ELSI Timeline First Release of ELSI Aug 2015 Jun 2016 May 2017

First Release of ELSI: May 2017 http://elsi-interchange.org Software Package - ELSI interface - Current ELPA, libomm, PEXSI Programming Language - Fortran 2003 - C/C++ interface Functionalities - Eigenvalues and eigenvectors - Density matrix - Chemical potential - Occupation numbers Portability - GNU, Intel, PGI, IBM, Cray - Cori, Edison, Mira, Theta, Titan Flexibility - 2 matrix I/O formats - 2 parallel modes - Concurrent instances - Tested in DGDFT, FHI-aims, NWChem (Global Arrays), SIESTA Scalability - (Hundreds of) thousands of MPI tasks 6/13

Supported Matrix Formats a) 2D block-cyclic distributed dense storage (BLACS standard) b) 1D block distributed compressed column storage (PEXSI standard) c) 1D block distributed compressed row storage supported implicitly due to symmetry Auto-conversion between (a) and (b) when necessary Example: BLACS input è PEXSI format è PEXSI solver è BLACS output 7/13

Performance of the Solvers (ELSI May 2017) DFT-PBE with FHI-aims Graphene monolayer (2D) From 1,800 to 7,200 atoms V. Yu et al., ELSI: A Unified Software Interface for Kohn-Sham Electronic Structure Solvers, arxiv:1705.11191 [physics.comp-ph]. Edison (Intel Ivy Bridge) 80 nodes x 24 CPU cores per node 14 numeric atom-centered orbitals per carbon atom PEXSI starts outperforming ELPA and libomm at 3,000 and 7,000 atoms, respectively 8/13

Performance of the Solvers (Today) DFT-PBE with FHI-aims Graphene monolayer (2D) From 800 to 7,200 atoms Edison (Intel Ivy Bridge) 80 nodes x 24 CPU cores per node 14 numeric atom-centered orbitals per carbon atom PEXSI starts outperforming ELPA and libomm at 3,000 800 and 7,000 3,200 atoms, respectively! Up-to-date 9/13

Enhanced Feature: Energy-Weighted Density Matrix ELPA libomm PEXSI DM * C () C () W(2I / S / )W * Im 4 w 6 () H z 6 + μ S EDM * C =() C =() W[ 2I / S / H / ]W * Im 4 w 6 =() : 6;< : 6;< H z 6 + μ S Pulay force (localized orbitals) Available with all three solvers Only computed when calling elsi_get_edm Geometry optimization 10/13

ELSI Timeline Electronic Structure Library Coding Workshop, ICTP, Trieste, 2017 Aug 2015 Jun 2016 May 2017 Jul 2017

Enhanced Feature: K-Point and Spin Calculations elsi_init(handle,...) New MPI communicator for communications among all k-points and spins! Set MPI communicator elsi_set_mpi(handle,my_mpi_comm) elsi_set_mpi_all(handle,mpi_comm_all)! Set spin and/or k-point elsi_set_spin(handle,n_spin,my_spin) elsi_set_kpt(handle,n_kpt,my_kpt,my_weight)! Compute density matrices elsi_dm_real(handle,my_h,my_s,my_dm,energy) elsi_finalize(handle) 11/13

Enhanced Feature: K-Point and Spin Calculations elsi_init(handle,...)! Set MPI communicator elsi_set_mpi(handle,my_mpi_comm) elsi_set_mpi_all(handle,mpi_comm_all) New Number of spins and my spin index! Set spin and/or k-point elsi_set_spin(handle,n_spin,my_spin) elsi_set_kpt(handle,n_kpt,my_kpt,my_weight)! Compute density matrices elsi_dm_real(handle,my_h,my_s,my_dm,energy) elsi_finalize(handle) 11/13

Enhanced Feature: K-Point and Spin Calculations elsi_init(handle,...)! Set MPI communicator elsi_set_mpi(handle,my_mpi_comm) elsi_set_mpi_all(handle,mpi_comm_all) New Number of k-points, my k-point index, and my k-point weight! Set spin and/or k-point elsi_set_spin(handle,n_spin,my_spin) elsi_set_kpt(handle,n_kpt,my_kpt,my_weight)! Compute density matrices elsi_dm_real(handle,my_h,my_s,my_dm,energy) elsi_finalize(handle) 11/13

Enhanced Feature: K-Point and Spin Calculations Minimal additions to the existing ELSI interface Capability to handle k-point and/or spin in parallel No overhead if not interested in k-point and/or spin elsi_init(handle,...)! Set MPI communicator elsi_set_mpi(handle,my_mpi_comm) elsi_set_mpi_all(handle,mpi_comm_all)! Set spin and/or k-point elsi_set_spin(handle,n_spin,my_spin) elsi_set_kpt(handle,n_kpt,my_kpt,my_weight) Spin K-point! Compute density matrices elsi_dm_real(handle,my_h,my_s,my_dm,energy) elsi_finalize(handle) 11/13

New Feature: CheSS and SIPs Solvers CheSS (density matrix) Fermi operator expansion with Chebyshev polynomials O(N) Both solvers connected to ELSI - SOLVER = 4 (CheSS) - SOLVER = 5 (SIPs) Flexibility of the ELSI infrastructure: Given compatible matrix formats, a new solver can be quickly added to ELSI in hours SIPs (eigensolution) Sparse, iterative eigensolver based on spectrum slicing O(N 2 ) Both are giving correct results Todo - Performance optimization - Benchmark O(N) solver Spectrum slicing 12/13

ELSI Timeline Aug 2015 Jun 2016 May 2017 Jul 2017 Future

Future Plans & Acknowledgments Performance-optimized, up-to-date solver libraries Iterative solvers, e.g. Davidson and CG solvers from Quantum Espresso ELSI decision layer for automatic, optimal choice of solvers More matrix formats used in KS- DFT codes to be supported Integration of ELSI into KS-DFT codes for production-level usage ELSI workshop in 2018 ELSI is an NSF SI2-SSI supported software infrastructure project under Grant Number 1450280. Any opinions, findings, and conclusions or recommendations expressed here are those of the author(s) and do not necessarily reflect the views of NSF. 13/13

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback