Polynomial filtering for interior eigenvalue problems Yousef Saad Department of Computer Science and Engineering University of Minnesota

Transcription

1 Polynomial filtering for interior eigenvalue problems Yousef Saad Department of Computer Science and Engineering University of Minnesota SIAM CSE Boston - March 1, 2013

2 First: Joint work with: Haw-ren Fang Grady Schoefield and Jim Chelikowsky [UT Austin] [windowing into PARSEC] Work supported in part by NSF (to 2012) and now by DOE 2 SIAM-CSE February 27, 2013

3 Introduction Q: A: How do you compute eigenvalues in the middle of the spectrum of a large Hermitian matrix? Method of choice: Shift and invert + some projection process (Lanczos, subspace iteration..) Main steps: 1) Select a shift (or sequence of shifts) σ; 2) Factor A σi: A σi = LDL T 3) Apply Lanczos algorithm to (A σi) 1 Solves with A σi carried out using factorization Limitation: factorization 3 SIAM-CSE February 27, 2013

4 Q: What if factoring A is too expensive (e.g., Large 3-D similation)? A: Obvious answer: Use iterative solvers... However: systems are highly indefinite Wont work too well. Digression: Still lots of work to do in iterative solution methods for highly indefinite linear systems 4 SIAM-CSE February 27, 2013

5 Other common characteristic: Need a very large number of eigenvalues and eigenvectors Applications: Excited states in quantum physics: TDDFT, GW,... Or just plain Density Functional Theory (DFT) Number of wanted eigenvectors is equal to number of occupied states [ == the number of valence electrons in DFT] An example: in real-space code (PARSEC), you can have a Hamiltonian of size a few Millions, and number of ev s in the tens of thousands. 5 SIAM-CSE February 27, 2013

6 Polynomial filtered Lanczos Possible solution: Use Lanczos with polynomial filtering. Idea not new (and not too popular in the past) 1. Very large problems; What is new? 2. (tens of) Thousands of eigenvalues; 3. Parallelism. Most important factor is 2. Main rationale of polynomial filtering : reduce the cost of orthogonalization Important application: compute the spectrum by pieces [ spectrum slicing a term coined by B. Parlett] 6 SIAM-CSE February 27, 2013

7 Introduction: What is filtered Lanczos? In short: we just replace (A σi) 1 in S.I. Lanczos by p k (A) where p k (t) = polynomial of degree k We want to compute eigenvalues near σ = 1 of a matrix A with Λ(A) [0, 4]. Use the simple transform: p 2 (t) = 1 α(t σ) 2. For α =.2, σ = 1 you get Use Lanczos with B = p 2 (A) Eigenvalues near σ become the dominant ones so Lanczos will work but they are now poorly separated slow convergence. 7 SIAM-CSE February 27, 2013

8 A low-pass filter [a,b]=[0,3], [ξ,η]=[0,1], d= base filter ψ(λ) polynomial filter ρ(λ) γ 0.6 f(λ) λ 8 SIAM-CSE February 27, 2013

9 A mid-pass filter [a,b]=[0,3], [ξ,η]=[0.5,1], d= base filter ψ(λ) polynomial filter ρ(λ) γ 0.6 f(λ) λ 9 SIAM-CSE February 27, 2013

10 Misconception: High degree polynomials are bad d=1000 Degree 1000 (zoom) f(λ) base filter ψ(λ) polynomial filter ρ(λ) γ λ d=1600 Degree 1600 (zoom) f(λ) base filter ψ(λ) polynomial filter ρ(λ) γ SIAM-CSE February 27, 2013 λ

11 Hypothetical scenario: large A, zillions of wanted e-values Assume A has size 10M (Not huge by todays standard)... and you want to compute 50,000 eigenvalues/vectors (huge for numerical analysits, not for physicists) in the lower part of the spectrum - or the middle. By (any) standard methods you will need to orthogonalize at least 50K vectors of size 10M Space is an issue: bytes = 4TB of mem *just for the basis* Orthogonalization is also an issue: = 50 PetaOPS. Toward the end, at step k, each orthogonalization step costs about 4kn 200, 000n for k close to 50, SIAM-CSE February 27, 2013

12 The alternative: Spectrum slicing or windowing Rationale. Eigenvectors on both ends of wanted spectrum need not be orthogonalized against each other : Idea: Get the spectrum by slices or windows Can get a few hundreds or thousands of vectors at a time. 12 SIAM-CSE February 27, 2013

13 Compute eigenpairs one slice at a time Deceivingly simple looking idea. Issues: Deal with interfaces : duplicate/missing eigenvalues Window size [need estimate of eigenvalues] polynomial degree 13 SIAM-CSE February 27, 2013

14 Spectrum slicing in PARSEC Being implemented in our code: Pseudopotential Algorithm for Real-Space Electronic Calcultions (PARSEC) See : A Spectrum Slicing Method for the Kohn-Sham Problem, G. Schofield, J. R. Chelikowsky and YS, Computer Physics Comm., vol 183, pp Refer to this paper for details on windowing and initial proof of concept 14 SIAM-CSE February 27, 2013

15 Computing the polynomials: Jackson-Chebyshev Chebyshev-Jackson approximation of a function f: f(x) k i=0 g k i γ it i (x) γ i = 2 δ i0 π x 2 f(x)dx δ i0 = Kronecker symbol The gi k s attenuate higher order terms in the sum. Attenuation coefficient gi k for k=50,100,150 0 g k i k=50 k=100 k= i (Degree) 15 SIAM-CSE February 27, 2013

16 Let α k = π k + 2, then : ( ) 1 i sin(α g k i = k+2 k ) cos(iα k ) + 1 k+2 cos(α k) sin(iα k ) sin(α k ) See Electronic structure calculations in plane-wave codes without diagonalization. Laurent O. Jay, Hanchul Kim, YS, and James R. Chelikowsky. Computer Physics Communications, 118:21 30, SIAM-CSE February 27, 2013

17 The expansion coefficients γ i When f(x) is a step function on [a, b] [ 1 1]: γ i = 1 (arccos(a) arccos(b)) : i = 0 π ( ) 2 sin(i arccos(a)) sin(i arccos(b)) : i > 0 π i A few examples follow 17 SIAM-CSE February 27, 2013

18 Computing the polynomials: Jackson-Chebyshev Polynomials of degree 30 for [a, b] = [.3,.6] Mid pass polynom. filter [ ]; Degree = 30 Standard Cheb. Jackson Cheb SIAM-CSE February 27, 2013

19 1.2 1 Mid pass polynom. filter [ ]; Degree = 80 Standard Cheb. Jackson Cheb SIAM-CSE February 27, 2013

20 1.2 1 Mid pass polynom. filter [ ]; Degree = 200 Standard Cheb. Jackson Cheb SIAM-CSE February 27, 2013

21 How to get the polynomial filter? Second approach Idea: φ First select an ideal filter e.g., a piecewise polynomial function a b For example φ = Hermite interpolating pol. in [0,a], and φ = 1 in [a, b] Referred to as the Base filter 21 SIAM-CSE February 27, 2013

22 Then approximate base filter by degree k polynomial in a least-squares sense. Can do this without numerical integration a b φ Main advantage: Extremely flexible. Method: Build a sequence of polynomials φ k which approximate the ideal PP filter φ, in the L 2 sense. Again 2 implementations 22 SIAM-CSE February 27, 2013

23 Define φ k the least-squares polynomial approximation to φ: φ k (t) = k φ, P j P j (t), j=1 where {P j } is a basis of polynomials that is orthonormal for some L 2 inner-product. Method 1: Use Stieljes procedure to computing orthogonal polynomials

24 ALGORITHM : 1 Stieljes 1. P 0 0, 2. β 1 = S 0, 3. P 1 (t) = 1 β 1 S 0 (t), 4. For j = 2,..., m Do 5. α j = t P j, P j, 6. S j (t) = t P j (t) α j P j (t) β j P j 1 (t), 7. β j+1 = S j, 8. P j+1 (t) = 1 β j+1 S j (t). 9. EndDo 24 SIAM-CSE February 27, 2013

25 Computation of Stieljes coefficients Problem: To compute the scalars α j and β j+1, of 3-term recurrence (Stieljes) + the expansion coefficients γ j. Need to avoid numerical integration. Solution: define orthogonal polynomials over two (or more) disjoint intervals see similar work YS 83: YS, Iterative solution of indefinite symmetric systems by methods using orthogonal polynomials over two disjoint intervals, SIAM Journal on Numerical Analysis, 20 (1983), pp E. KOKIOPOULOU AND YS, Polynomial Filtering in Latent Semantic Indexing for Information Retrieval, in Proc. ACM-SIGIR Conference on research and development in information retrieval, Sheffield, UK, (2004) 25 SIAM-CSE February 27, 2013

26 Let large interval be [0, b] should contain Λ(B) Assume 2 subintervals. On subinterval [a l 1, a l ], l = 1, 2 define the inner-product ψ 1, ψ 2 al 1,a l by ψ 1, ψ 2 al 1,a l = al a l 1 ψ 1 (t)ψ 2 (t) (t al 1 )(a l t) dt. Then define the inner product on [0, b] by ψ 1, ψ 2 = a 0 ψ 1 (t)ψ 2 (t) t(a t) dt + ρ b a ψ 1 (t)ψ 2 (t) (t a)(b t) dt. To avoid numerical integration, use a basis of Chebyshev polynomials on interval [YS 83] 26 SIAM-CSE February 27, 2013

27 Mehod 2 : Filtered CG/CR - like polynomial iterations Want: a CG-like (or CR-like) algorithms for which the inderlying residual polynomial or solution polynomial are Least-squares filter polynomials Seek s to minimize φ(λ) λ s(λ) w with respect to a certain norm. w. Equivalently, minimize (1 φ) (1 λ s(λ)) w over all polynomials s of degree k. Focus on second view-point (residual polynomial) goal is to make r(λ) 1 λs(λ) close to 1 φ. 27 SIAM-CSE February 27, 2013

28 Recall: Conjugate Residual Algorithm ALGORITHM : 2 Conjugate Residual Algorithm 1. Compute r 0 := b Ax 0, p 0 := r 0 2. For j = 0, 1,..., until convergence Do: 3. α j := (r j, Ar j )/(Ap j, Ap j ) 4. x j+1 := x j + α j p j 5. r j+1 := r j α j Ap j 6. β j := (r j+1, Ar j+1 )/(r j, Ar j ) 7. p j+1 := r j+1 + β j p j 8. Compute Ap j+1 = Ar j+1 + β j Ap j 9. EndDo Think in terms of polynomial iteration 28 SIAM-CSE February 27, 2013

29 ALGORITHM : 3 Filtered CR polynomial Iteration 1. Compute r 0 := b Ax 0, p 0 := r 0 π 0 = ρ 0 = 1 1.a. Compute λπ 0 2. For j = 0, 1,..., until convergence Do: 3. α j :=< ρ j, λρ j > w / < λπ j, λπ j > w 3.a. α j := α j < 1 φ, λπ j > w / < λπ j, λπ j > w 4. x j+1 := x j + α j p j 5. r j+1 := r j α j Ap j ρ j+1 = ρ j α j λπ j 6. β j :=< ρ j+1, λρ j+1 > w / < ρ j, λρ j > w 7. p j+1 := r j+1 + β j p j π j+1 := ρ j+1 + β j π j 8. Compute λπ j+1 9. EndDo All polynomials expressed in Chebyshev basis - cost of algorithm is negligible [ O(k 2 ) for deg. k. ] 29 SIAM-CSE February 27, 2013

30 A few mid-pass filters of various degrees Four examples of middle-pass filters ψ(λ) and their polynomial approximations ρ(λ). Degrees 20 and 30 d=20 d=30 1 base filter ψ(λ) polynomial filter ρ(λ) γ 1 base filter ψ(λ) polynomial filter ρ(λ) γ f(λ) 0.4 f(λ) λ λ 30 SIAM-CSE February 27, 2013

31 Degrees 50 and 100 d=50 d= base filter ψ(λ) polynomial filter ρ(λ) γ base filter ψ(λ) polynomial filter ρ(λ) γ f(λ) 0.4 f(λ) λ λ 31 SIAM-CSE February 27, 2013

32 Base Filter to build a Mid-Pass filter polynomial We partition [0, b] into five sub-intervals, [0, b] = [0, 1][τ 1, τ 2 ] [τ 2, τ 3 ] [τ 3, τ 4 ] [τ 4, b] 0 τ τ τ τ b Set: ψ(t) = 0 in [0, τ 1 ] [τ 4, b] and ψ(t) = 1 in [τ 2, τ 3 ] Use standard Hermite interpolation to get brigde functions in [τ 1, τ 2 ] and [τ 3, τ 4 ] 32 SIAM-CSE February 27, 2013

33 References A Filtered Lanczos Procedure for Extreme and Interior Eigenvalue Problems, H. R. Fang and YS, SISC 34(4) A (2012). For details on window-less implementation (one slice) + code Computation of Large Invariant Subspaces Using Polynomial Filtered Lanczos Iterations with Applications in Density Functional Theory, C. Bekas and E. Kokiopoulou and YS, SIMAX 30(1), (2008). Filtered Conjugate Residual-type Algorithms with Applications, YS; SIMAX 28 pp (2006) 33 SIAM-CSE February 27, 2013

34 Tests Test matrices Experiments performed in sequential mode: on two dualcore AMD Opteron(tm) Processors 2.2GHz and 16GB memory. Test matrices: * Five Hamiltonians from electronic structure calculations, * An integer matrix named Andrews, and * A discretized Laplacian (FD) 34 SIAM-CSE February 27, 2013

35 nz = SIAM-CSE February 27, 2013

36 Matrix characteristics matrix n nnz nnz n full eigen-range Fermi [a, b] n 0 GE87H76 112,985 7,892, [ , ] 212 Ge99H ,985 8,451, [ , ] 248 SI41Ge41H72 185,639 15,011, [ , ] 200 Si87H76 240,369 10,661, [ , ] 212 Ga41As41H72 268,096 18,488, [ , ] 200 Andrews 60, , [0, ] N/A Laplacian 1,000,000 6,940, [ , ] N/A 36 SIAM-CSE February 27, 2013

37 Experimental set-up eigen-interval # eig # eig matrix [ξ, η] in [ξ, η] in [a, η] η ξ b a η a b a GE87H76 [ 0.645, ] Ge99H100 [ 0.65, ] SI41Ge41H72 [ 0.64, ] Si87H76 [ 0.66, 0.33] Ga41As41H72 [ 0.64, 0.0] Andrews [4, 5] 1,844 3, Laplacian [1, 1.01] 276 >17, SIAM-CSE February 27, 2013

38 Results for Ge99H100 -set 1 of stats method degree # iter # matvecs memory d = 20 1,020 20,400 1,117 filt. Lan. d = , (high-pass) d = , d = , d = , filt. Lan. d = , (low-pass) d = , d = , Part. Lanczos 5,140 5,140 4,883 ARPACK 6,233 6,233 1, SIAM-CSE February 27, 2013

39 Results for Ge99H100 -CPU times (sec.) method degree ρ(a)v reorth eigvec total d = 20 1, ,417 filt. Lan. d = 30 1, ,440 (high-pass) d = 50 1, ,479 d = 100 1, ,930 d = filt. Lan. d = (low-pass) d = 30 1, ,123 d = 50 1, ,342 Part. Lanczos 234 1, ,962 ARPACK , , SIAM-CSE February 27, 2013

40 Results for Andrews - set 1 of stats method degree # iter # matvecs memory d = 20 9, ,800 4,829 filt. Lan. d = 30 6, ,120 2,799 (mid-pass) d = 50 3, ,000 1,947 d = 100 2, ,000 1,131 d = 10 5,990 59,900 2,799 filt. Lan. d = 20 4,780 95,600 2,334 (high-pass) d = 30 4, ,800 2,334 d = 50 4, ,500 2,334 Part. Lanczos 22,345 22,345 10,312 ARPACK 30,716 30,716 6, SIAM-CSE February 27, 2013

41 Results for Andrews - CPU times (sec.) method degree ρ(a)v reorth eigvec total d = 20 2, ,834 9,840 filt. Lan. d = 30 2, ,151 5,279 (mid-pass) d = 50 3, ,810 d = 100 3, ,147 d = 10 1,152 2,911 2,391 7,050 filt. Lan. d = 20 1,335 1,718 1,472 4,874 (high-pass) d = 30 1,806 1,218 1,274 4,576 d = 50 3,187 1,032 1,383 5,918 Part. Lanczos ,455 64, ,664 ARPACK ,492 18, , SIAM-CSE February 27, 2013

42 Results for Laplacian set 1 of stats method degree # iter # matvecs memory 600 1, ,000 10,913 mid-pass filter 1, ,000 7,640 1, ,136,000 6, SIAM-CSE February 27, 2013

43 Results for Laplacian CPU times method degree ρ(a)v reorth eigvec total , ,279 mid-pass filter 1, , ,384 1, , , SIAM-CSE February 27, 2013

44 Conclusion Quite appealing general approach when number of eigenvectors to be computed is large and when Matvec is not too expensise Will not work too well for generalized eigenvalue problem Code available here 44 SIAM-CSE February 27, 2013