Localized Optimization: Exploiting non-orthogonality to efficiently minimize the Kohn-Sham Energy

Transcription

1 Localized Optimization: Exploiting non-orthogonality to efficiently minimize the Kohn-Sham Energy Courant Institute of Mathematical Sciences, NYU Lawrence Berkeley National Lab 16 December 2009

2 Joint work with Michael Overton NYU Juan Meza LBNL Chao Yang LBNL February NC State, Raleigh NC

3

4 In the past Every attempt to employ mathematical methods in the study of chemical questions must be considered profoundly irrational and contrary to the spirit of chemistry. If mathematical analysis should ever hold a prominent place in chemistry - an aberration which is happily almost impossible - it would occasion a rapid and widespread degeneration of that science. Auguste Comte, 1830

5 In the future Whereas computational methods nowadays mainly supplement experimentally obtained information, they are expected increasingly to supersede this information. Stephen Goedecker, 1999

6

7 CdSe Solar Cell CMOS electrons > 1,000,000 electrons > 50,000 electrons

8

9 Many-body Schrödinger Equation H Ψ ( r 1, r 2,..., r ne ) = λψ ( r 1, r 2,..., r ne ) If the n u nuclei are fixed in space at positions ˆr j, n e H = - 1 Δ ri 2 1 n u j=1 n e i=1 z j r i ˆr j + 1 r 1 i,j n i r j e

10 Many-body Schrödinger Equation H Ψ ( r 1, r 2,..., r ne ) = λψ ( r 1, r 2,..., r ne ) If the n u nuclei are fixed in space at positions ˆr j, n e H = - 1 Δ ri 2 1 n u j=1 n e i=1 z j r i ˆr j + 1 r 1 i,j n i r j e Ψ contains all the information in the system and satisfies orthogonality and antisymmetry properties

11 Many-body Schrödinger Equation H Ψ ( r 1, r 2,..., r ne ) = λψ ( r 1, r 2,..., r ne ) If the n u nuclei are fixed in space at positions ˆr j, n e H = - 1 Δ ri 2 1 n u j=1 n e i=1 z j r i ˆr j + 1 r 1 i,j n i r j e Ψ contains all the information in the system and satisfies orthogonality and antisymmetry properties Ψ ( r 1, r 2,..., r ne ) 2 dr 1... dr ne gives probability density

12 Many-body Schrödinger Equation H Ψ ( r 1, r 2,..., r ne ) = λψ ( r 1, r 2,..., r ne ) If the n u nuclei are fixed in space at positions ˆr j, n e H = - 1 Δ ri 2 1 n u j=1 n e i=1 z j r i ˆr j + 1 r 1 i,j n i r j e Ψ contains all the information in the system and satisfies orthogonality and antisymmetry properties Ψ ( r 1, r 2,..., r ne ) 2 dr 1... dr ne gives probability density λ represents the quantized energy

13 Many-body Schrödinger Equation H Ψ ( r 1, r 2,..., r ne ) = λψ ( r 1, r 2,..., r ne ) If the n u nuclei are fixed in space at positions ˆr j, n e H = - 1 Δ ri 2 1 n u j=1 n e i=1 z j r i ˆr j + 1 r 1 i,j n i r j e Ψ contains all the information in the system and satisfies orthogonality and antisymmetry properties Ψ ( r 1, r 2,..., r ne ) 2 dr 1... dr ne gives probability density λ represents the quantized energy If we attempt to solve on a 32 x 32 x 32 grid, and n e = 5, H has dimension

14

15 Density Functional Theory Hohenberg and Kohn 1964: Proved that at ground state, total energy is function of charge density ρ ( r) = n e Ψ* ( r, r 2,..., r ne ) Ψ ( r, r 2,..., r ne ) dr 2 dr 3... dr ne

16 Density Functional Theory Hohenberg and Kohn 1964: Proved that at ground state, total energy is function of charge density ρ ( r) = n e Ψ* ( r, r 2,..., r ne ) Ψ ( r, r 2,..., r ne ) dr 2 dr 3... dr ne Kohn and Sham 1965: Proposed a practical formulation, using n e single-particle, orthogonal wavefunctions ψ j that do not interact. Electron-electron interactions modeled by exchange-correlation energy.

17 The Kohn-Sham Equations Charge Density: n e ρ (r) = ψ* i (r ) ψ i ( r ) 1

18 The Kohn-Sham Equations Charge Density: n e ρ (r) = ψ* i (r ) ψ i ( r ) Exchange-correlation energy: E xc (ρ) = ρ (r ) ε xc [ρ ( r )]dr 1

19 The Kohn-Sham Equations Charge Density: n e ρ (r) = ψ* i (r ) ψ i ( r ) Exchange-correlation energy: E xc (ρ) = ρ (r ) ε xc [ρ ( r )]dr 1 Kohn-Sham energy: n e E KS [{ψ j }] = ψ j ( Δψ j ) dr + V ion ( r ) ρ ( r ) dr ρ ( r ) ρ ( r ) r r drdr + E xc ( ρ ).

20 Kohn-Sham Equations continued min E KS ( ψ j ) s.t. ψi * ψ j = δ i,j for i = 1, 2,..., n e

21 Kohn-Sham Equations continued KKT Conditions: min E KS ( ψ j ) s.t. ψi * ψ j = δ i,j for i = 1, 2,..., n e H KS ( ρ ) ψ i = λ i ψ i, ψ j * ψ j = δ i,j, for i = 1, 2,..., n e where H KS = Δ + V ion ( r ) +ρ (r ) * 1 r + V xc ( ρ) = Δ + V ( ρ (r) )

22 Non-orthogonal Kohn-Sham Equations

23 Non-orthogonal Kohn-Sham Equations Overlap matrix: S jk = ψ* j ψ k dr Charge-density: ρ (r) = j ψ* j (r ) (S -1 ) jk ψ k ( r) k

24 Non-orthogonal Kohn-Sham Equations Overlap matrix: S jk = ψ* j ψ k dr Charge-density: ρ (r) = j ψ* j (r ) (S -1 ) jk ψ k ( r) k Kohn-Sham Energy: E KS ( {ψ j }) = ( S -1 ) jk j k ψ j ( Δψ k ) dr + V ion ( r ) ρ ( r ) dr ρ ( r ) ρ ( r ) r r drdr + E xc ( ρ)

25 Non-orthogonal Kohn-Sham Equations Overlap matrix: S jk = ψ* j ψ k dr Charge-density: ρ (r) = j ψ* j (r ) (S -1 ) jk ψ k ( r) k Kohn-Sham Energy: E KS ( {ψ j }) = ( S -1 ) jk j k ψ j ( Δψ k ) dr + V ion ( r ) ρ ( r ) dr ρ ( r ) ρ ( r ) r r drdr + E xc ( ρ) This energy is independent of basis

26

27 Numerical Methods Step 1: Choose a discretization

28 Numerical Methods Step 1: Choose a discretization Plane-waves basis: ψ ( r) = k α k e ig k r 1 Advantages: ρ ( r ) * r Matrix Multiplication (FFT) Disadvantages: Inherently nonlocal

29 Numerical Methods Step 1: Choose a discretization Plane-waves basis: ψ ( r) = k α k e ig k r 1 Advantages: ρ ( r ) * r Matrix Multiplication (FFT) Disadvantages: Inherently nonlocal Finite-differences: ψ ( r j ) = 1 h [ψ ( r j + h ) ψ ( r j h)] Advantages: Localized, highly sparse 1 Disadvantages:ρ ( r ) * r requires solution to a Poisson problem

30 Discretized Kohn-Sham Equations E KS (X) = E kineticionic ( X ) +E Hartree ( X ) +E xc ( X) ( ( E kineticionic = trace (X*X) -1 X* 1 ) ) 2 L + V ion X E Hartree = 1 2 ρt L ρ E xc = ρ T ε xc [ρ] [ ρ (X) = diag X ( X*X ) -1 ] X*

31 Discretized Kohn-Sham Equations E KS (X) = E kineticionic ( X ) +E Hartree ( X ) +E xc ( X) ( ( E kineticionic = trace (X*X) -1 X* 1 ) ) 2 L + V ion X E Hartree = 1 2 ρt L ρ E xc = ρ T ε xc [ρ] [ ρ (X) = diag X ( X*X ) -1 ] X* E (XG) = E (X)

32 Simpli ed Kohn-Sham Equations (The Model Problem) ( ( E (X) = trace (X*X ) -1 1 X* 2 L + α ) ) 1 2 V X + α 2 4 ρt L -1 ρ [ ρ = diag X ( X*X ) -1 ] X*

33 Simpli ed Kohn-Sham Equations (The Model Problem) ( ( E (X) = trace (X*X ) -1 1 X* 2 L + α ) ) 1 2 V X + α 2 4 ρt L -1 ρ Look Familiar? [ ρ = diag X ( X*X ) -1 ] X*

34 Simpli ed Kohn-Sham Equations (The Model Problem) ( ( E (X) = trace (X*X ) -1 1 X* 2 L + α ) ) 1 2 V X + α 2 4 ρt L -1 ρ [ ρ = diag X ( X*X ) -1 ] X* Look Familiar? x*ax ( x*x trace (X*X ) -1 ) X*AX

35 Numerical Methods Step 1: Choose a discretization Step 2: Compute a solution

36 Numerical Methods Step 1: Choose a discretization Step 2: Compute a solution SCF: Solve the nonlinear Kohn-Sham equations

37 Numerical Methods Step 1: Choose a discretization Step 2: Compute a solution SCF: Solve the nonlinear Kohn-Sham equations CG: Directly minimize the Kohn-Sham Energy

38 Self-Consistent Field Iteration [ ½Δ + V(ρ(r))]ψ = Eψ i i i ρ(r) = ψ ² i V(ρ(r))

39 Self-Consistent Field Iteration [ ½Δ + V(ρ(r))]ψ = Eψ i i i O ( n 3 e ) ρ(r) = ψ ² i V(ρ(r))

40 Self-Consistent Field Iteration [ ½Δ + V(ρ(r))]ψ = Eψ ρ(r) = ψ ² i i i i O ( n 3 e ) Convergence slow, non-monotonic V(ρ(r))

41 Self-Consistent Field Iteration [ ½Δ + V(ρ(r))]ψ = Eψ ρ(r) = ψ ² i i i i O ( n 3 e ) Convergence slow, non-monotonic Charge-mixing to accelerate convergence V(ρ(r))

42 CG (Fletcher-Reeves) 1. RM = - X E ( X 0 ) 2. PM = RM 3. XM = X 0 4. repeat until convergence criteria are satisfied 5. Choose αm from a line search on E ( XM + αpm) 6. X k+1 = XM + αmpm 7. R k+1 = - X E ( X k+1 ) 8. β k+1 = (rk+1 ) T r k+1 (r k ) T r k 9. P k+1 = R k+1 + β k+1 ( PM) 10. k = k end loop

43

44 Two Equal Solutions Orthogonal Solution A Localized Solution Student Version of MATLAB Student Version of MATLAB

45 Simpli ed Kohn-Sham Equations (The Model Problem) ( ( E (X) = trace (X*X ) -1 1 X* 2 L + α ) ) 1 2 V X + α 2 4 ρt L -1 ρ [ ρ = diag X ( X*X ) -1 ] X*

46 Localization and Sparsity nz = nz = 150

47

48 Maintaining Localization/sparsity An iterative algorithm need not maintain the sparsity pattern.

49 Maintaining Localization/sparsity An iterative algorithm need not maintain the sparsity pattern. Naive approach: Truncate, i.e. set X k+1 = (X k+1 ) T

50 Maintaining Localization/sparsity An iterative algorithm need not maintain the sparsity pattern. Naive approach: Truncate, i.e. set X k+1 = (X k+1 ) T Before truncation After truncation Student Version of MATLAB Student Version of MATLAB

51 Maintaining Localization/sparsity An iterative algorithm need not maintain the sparsity pattern. Naive approach: Truncate, i.e. set X k+1 = (X k+1 ) T E and Gao approach: Minimize truncation error (localize)

52 Maintaining Localization/sparsity An iterative algorithm need not maintain the sparsity pattern. Naive approach: Truncate, i.e. set X k+1 = (X k+1 ) T E and Gao approach: Minimize truncation error (localize) Compute invertible G which minimizes XG (XG) T F Remember E (XG) = E (X) for any invertible G

53 Sparse Subspace E((XMG T )) E(XMG) E(XMG) = E(XM) (XMG) T XM

54 CG + Localization (CG L by E and Gao) 1. RM = - X E ( X 0 ) 2. PM = RM 3. XM = X 0 4. repeat until convergence criteria are satisfied 5. Choose αm from a line search on E ( XM + αpm) 6. X k+1 = XM + αmpm 7. G = arg min G X k+1 G (X k+1 G ) T 8. X k+1 = (X k+1 G) T 9. R k+1 = - X E ( X k+1 ) 10. β k+1 = (rk+1 ) T r k+1 (r k ) T r k 11. P k+1 = R k+1 + β k+1 ( PMG) T 12. k = k end loop

55 Avoiding Local Minima: CG L versus truncation n = 100, n e = localized truncated

56 CG + Localization (CG L by E and Gao) 1. RM = - X E ( X 0 ) 2. PM = RM 3. XM = X 0 4. repeat until convergence criteria are satisfied 5. Choose αm from a line search on E ( XM + αpm) 6. X k+1 = XM + αmpm 7. G = arg min G X k+1 G (X k+1 G ) T 8. X k+1 = (X k+1 G) T 9. R k+1 = - X E ( X k+1 ) 10. β k+1 = (rk+1 ) T r k+1 (r k ) T r k 11. P k+1 = R k+1 + β k+1 ( PMG) T 12. k = k end loop

57 The Plan Develop an optimization algorithm that

58 The Plan Develop an optimization algorithm that maintains the property of avoiding local minima

59 The Plan Develop an optimization algorithm that maintains the property of avoiding local minima evaluates the energy only at sparse iterates.

60 LocOpt: Localize the search direction { Sparse Dense { Pk { Sparse + Pk Localization (( + Pk)Gk) T

61 LocOptCG 1. RM = - X E ( X 0 ) 2. PM = RM 3. XM = X 0 4. repeat until convergence criteria are satisfied 5. G = arg min G ( XM + PM ) G (( XM + PM ) G ) T 6. PM = (( XM + PM ) G ) T XM 7. Choose αm from a line search on E ( XM + α PM) 8. X k+1 = XM + αm PM 9. R k+1 = - X E ( X k+1 ) 10. β k+1 = (rk+1 ) T r k+1 (r k ) T r k 11. P k+1 = R k+1 + β k+1 PM 12. k = k end loop

62 Avoiding Local Minima: LocOptCG vs: CG L!0.625! E and Gao o - Our method!0.635!0.64!0.645!0.65! Tolerance = 10 ⁷; n e = 5, n = 100

63 Monotonic Convergence !1 - E and Gao - Our method log(e - E*) 10!2 10!3 10!4 10!5 10!6 10!7 10!8 10! n e = 5, n = 100

64 A real problem: Methane (CH₄)

65 A real problem: Methane (CH₄) 8 Electrons in 4 Orbitals

66 A real problem: Methane (CH₄) 8 Electrons in 4 Orbitals Hamiltonian dimension 5832x5832

67 A real problem: Methane (CH₄) 8 Electrons in 4 Orbitals Hamiltonian dimension 5832x5832 Known solution from : Ry

68 Methane (CH₄) Solution

69 Methane (CH₄) Solution nz = 2762

70 Methane (CH₄) Solution CG LMBFGS nz = 2762 Energy: Ry

71 Methane (CH₄) Solution (cont) LocOpt: Rho -RSDFT: Rho

72 A Variation for { Sparse Dense { + PM PM + PM)GM { Sparse Localization (( ) T (( + PM)GM) T E T

73 Something magic happens 10 2 log(em (-17.62))) LMBFGS + old CG + new LMBFGS + new

74

75 Scale & Conditioning E (XG) = E (X)

76 Scale & Conditioning E (XG) = E (X) In particular, what if G = Diag[α 1, α 2,..., α ne ]?

77 Scale & Conditioning E (XG) = E (X) In particular, what if G = Diag[α 1, α 2,..., α ne ]? Can we normalize columns?

78 Scale & Conditioning E (XG) = E (X) In particular, what if G = Diag[α 1, α 2,..., α ne ]? Can we normalize columns? How does this affect search direction scale?

79 Localization & Descent min s.t. ( XM + PM ) G (( XM + PM ) G ) T F G is invertible

80 Localization & Descent min s.t. ( XM + PM ) G (( XM + PM ) G ) T F G is invertible ( trace E T P ) < 0

81 Localization & Descent min s.t. ( XM + PM ) G (( XM + PM ) G ) T F G is invertible ( trace E T P ) ( trace ( E ) T ( )) T ( XM + PM ) G < 0.

82 Localization & Descent min s.t. ( XM + PM ) G (( XM + PM ) G ) T F G is invertible ( trace ( E ) T ( )) T ( XM + PM ) G < 0

83 The Gradient E KS (X) = E kineticionic ( X ) +E Hartree ( X ) +E xc ( X) E kineticionic -XS -1 X* ( L + V ) XS -1 + (L + V ) XS -1 E Hartree Diag ( L -1 ρ ) XS -1 XS -1 X* Diag ( L -1 ρ ) XS -1 E xc... ρ (X) = diag [ X ( X*X ) -1 ] X*

84 The Overlap Matrix x S S -1 Student Version of MATLAB Student Version of MATLAB

85

86 Integration with BFGS (LMBFGS) 10 2 CG !2 BFGS 10!4 10!6 10!8 10!10 10!12 10!14 10! n e = 5, n = 100. Bad Search Directions: CG (0), BFGS (7)

87 Solve (more) realistic problems Based on by Chelikowsky, Troullier, and Saad

88 Solve (more) realistic problems Based on by Chelikowsky, Troullier, and Saad Open-source Matlab code

89 Solve (more) realistic problems Based on by Chelikowsky, Troullier, and Saad Open-source Matlab code In real-space, using finite-differences

90 Solve (more) realistic problems Based on by Chelikowsky, Troullier, and Saad Open-source Matlab code In real-space, using finite-differences Limited in problem size and mesh length

91 Solve (more) realistic problems Based on by Chelikowsky, Troullier, and Saad Open-source Matlab code In real-space, using finite-differences Limited in problem size and mesh length Just started designing and writing a object-oriented C/C++ implementation to run on machines

92