High-performance computation of pseudospectra

Transcription

1 1 / 80 High-performance computation of pseudospectra Jack Poulson Department of Mathematics Stanford University Innovative Computing Laboratory The University of Tennessee October 24, 2014

2 A note on collaborators 2 / 80 Multishift Hessenberg solves: collaboration with Greg Henry

3 3 / 80 Overview of talk Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

4 4 / 80 Overview of talk Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

5 5 / 80 Overview of talk Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

6 6 / 80 Overview of talk Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

7 7 / 80 Overview of talk Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

8 8 / 80 Overview of talk Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

9 9 / 80 Overview of talk Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

10 Outline 10 / 80 Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

11 11 / 80 Van Loan s algorithm and the Demmel matrix Efficient algorithm for evaluating resolvent over vertical line in complex plane [Van Loan-1985] Relationship to nearest stable matrix based on false conjecture, counter-example given in [Demmel-1987] Counter-example was first pseudospectral (computer) plot

12 12 / 80 Van Loan s algorithm and the Demmel matrix Efficient algorithm for evaluating resolvent over vertical line in complex plane [Van Loan-1985] Relationship to nearest stable matrix based on false conjecture, counter-example given in [Demmel-1987] Counter-example was first pseudospectral (computer) plot

13 13 / 80 Van Loan s algorithm and the Demmel matrix Efficient algorithm for evaluating resolvent over vertical line in complex plane [Van Loan-1985] Relationship to nearest stable matrix based on false conjecture, counter-example given in [Demmel-1987] Counter-example was first pseudospectral (computer) plot

14 Van Loan s algorithm and the Demmel matrix 14 / 80 Efficient algorithm for evaluating resolvent over vertical line in complex plane [Van Loan-1985] Relationship to nearest stable matrix based on false conjecture, counter-example given in [Demmel-1987] Counter-example was first pseudospectral (computer) plot D(n, β) = (β J β 1,n ) 1, n = 3, β = 100

15 Outline 15 / 80 Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

16 Pseudospectra 16 / 80 With convention (λi A) 1 p = for λ L(A), eigenvalues are singularities of resolvent norm (ξi A) 1 p. Natural generalization of spectrum for each p-norm and ɛ > 0: { L p ɛ (A) = ξ C : (ξi A) 1 p > 1 } ɛ Reinvented many times [Varah-1967,Landau-1975,Godunov et al.-1982,trefethen-1990,hinrichsen/pritchard-1992,...] Extensive review of field in Trefethen and Embree s book Spectra and pseudospectra... Most common tool is EigTool [Wright et al.-2001], but level 2 BLAS (trsv) and sequential

17 Pseudospectra 17 / 80 With convention (λi A) 1 p = for λ L(A), eigenvalues are singularities of resolvent norm (ξi A) 1 p. Natural generalization of spectrum for each p-norm and ɛ > 0: { L p ɛ (A) = ξ C : (ξi A) 1 p > 1 } ɛ Reinvented many times [Varah-1967,Landau-1975,Godunov et al.-1982,trefethen-1990,hinrichsen/pritchard-1992,...] Extensive review of field in Trefethen and Embree s book Spectra and pseudospectra... Most common tool is EigTool [Wright et al.-2001], but level 2 BLAS (trsv) and sequential

18 Pseudospectra 18 / 80 With convention (λi A) 1 p = for λ L(A), eigenvalues are singularities of resolvent norm (ξi A) 1 p. Natural generalization of spectrum for each p-norm and ɛ > 0: { L p ɛ (A) = ξ C : (ξi A) 1 p > 1 } ɛ Reinvented many times [Varah-1967,Landau-1975,Godunov et al.-1982,trefethen-1990,hinrichsen/pritchard-1992,...] Extensive review of field in Trefethen and Embree s book Spectra and pseudospectra... Most common tool is EigTool [Wright et al.-2001], but level 2 BLAS (trsv) and sequential

19 Pseudospectra 19 / 80 With convention (λi A) 1 p = for λ L(A), eigenvalues are singularities of resolvent norm (ξi A) 1 p. Natural generalization of spectrum for each p-norm and ɛ > 0: { L p ɛ (A) = ξ C : (ξi A) 1 p > 1 } ɛ Reinvented many times [Varah-1967,Landau-1975,Godunov et al.-1982,trefethen-1990,hinrichsen/pritchard-1992,...] Extensive review of field in Trefethen and Embree s book Spectra and pseudospectra... Most common tool is EigTool [Wright et al.-2001], but level 2 BLAS (trsv) and sequential

20 Pseudospectra 20 / 80 With convention (λi A) 1 p = for λ L(A), eigenvalues are singularities of resolvent norm (ξi A) 1 p. Natural generalization of spectrum for each p-norm and ɛ > 0: { L p ɛ (A) = ξ C : (ξi A) 1 p > 1 } ɛ Reinvented many times [Varah-1967,Landau-1975,Godunov et al.-1982,trefethen-1990,hinrichsen/pritchard-1992,...] Extensive review of field in Trefethen and Embree s book Spectra and pseudospectra... Most common tool is EigTool [Wright et al.-2001], but level 2 BLAS (trsv) and sequential

21 Outline 21 / 80 Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

22 22 / 80 Relationship to previous work Embarrassingly parallel Lanczos [Braconnier-1996] Parallel sparse-direct shift-and-invert [Fraysse et al.-1996] Parallel path-following methods, e.g., [Mezher-2001,Bekas et al.-2000,2001,...] Our focus on level 3 (parallel) batch evaluation over point clouds after parallel Schur decomposition Key idea: simultaneously drive many Lanczos/Arnoldi methods via multishift extensions of standard TRSM algorithm Software for one and two-norm pseudospectra already released in Elemental [P. et al.-2013]

23 23 / 80 Relationship to previous work Embarrassingly parallel Lanczos [Braconnier-1996] Parallel sparse-direct shift-and-invert [Fraysse et al.-1996] Parallel path-following methods, e.g., [Mezher-2001,Bekas et al.-2000,2001,...] Our focus on level 3 (parallel) batch evaluation over point clouds after parallel Schur decomposition Key idea: simultaneously drive many Lanczos/Arnoldi methods via multishift extensions of standard TRSM algorithm Software for one and two-norm pseudospectra already released in Elemental [P. et al.-2013]

24 24 / 80 Relationship to previous work Embarrassingly parallel Lanczos [Braconnier-1996] Parallel sparse-direct shift-and-invert [Fraysse et al.-1996] Parallel path-following methods, e.g., [Mezher-2001,Bekas et al.-2000,2001,...] Our focus on level 3 (parallel) batch evaluation over point clouds after parallel Schur decomposition Key idea: simultaneously drive many Lanczos/Arnoldi methods via multishift extensions of standard TRSM algorithm Software for one and two-norm pseudospectra already released in Elemental [P. et al.-2013]

25 25 / 80 Relationship to previous work Embarrassingly parallel Lanczos [Braconnier-1996] Parallel sparse-direct shift-and-invert [Fraysse et al.-1996] Parallel path-following methods, e.g., [Mezher-2001,Bekas et al.-2000,2001,...] Our focus on level 3 (parallel) batch evaluation over point clouds after parallel Schur decomposition Key idea: simultaneously drive many Lanczos/Arnoldi methods via multishift extensions of standard TRSM algorithm Software for one and two-norm pseudospectra already released in Elemental [P. et al.-2013]

26 26 / 80 Relationship to previous work Embarrassingly parallel Lanczos [Braconnier-1996] Parallel sparse-direct shift-and-invert [Fraysse et al.-1996] Parallel path-following methods, e.g., [Mezher-2001,Bekas et al.-2000,2001,...] Our focus on level 3 (parallel) batch evaluation over point clouds after parallel Schur decomposition Key idea: simultaneously drive many Lanczos/Arnoldi methods via multishift extensions of standard TRSM algorithm Software for one and two-norm pseudospectra already released in Elemental [P. et al.-2013]

27 27 / 80 Relationship to previous work Embarrassingly parallel Lanczos [Braconnier-1996] Parallel sparse-direct shift-and-invert [Fraysse et al.-1996] Parallel path-following methods, e.g., [Mezher-2001,Bekas et al.-2000,2001,...] Our focus on level 3 (parallel) batch evaluation over point clouds after parallel Schur decomposition Key idea: simultaneously drive many Lanczos/Arnoldi methods via multishift extensions of standard TRSM algorithm Software for one and two-norm pseudospectra already released in Elemental [P. et al.-2013]

28 (Extended) Van Loan algorithm [Van Loan-1985,Lui-1997] Given a reduction to condensed form, A = QGQ H, { L p ɛ (A) = ξ C : Q(ξI G) 1 Q H p > 1 }, ɛ and, when p = 2, L 2 ɛ (A) = { ξ C : (ξi G) 1 2 > 1 }, ɛ First reduce to (quasi-)triangular/hessenberg form, then, (Restarted) Lanczos/Arnoldi for each (ξi G) 1 2, or Blocked Hager [Higham/Tisseur-2000] for each Q(ξI G) 1 Q H 1. Note that this algorithm makes less sense when (ξi A) 1 can already be applied in quadratic time (e.g., if A is 3D FEM discretization) 28 / 80

29 (Extended) Van Loan algorithm [Van Loan-1985,Lui-1997] Given a reduction to condensed form, A = QGQ H, { L p ɛ (A) = ξ C : Q(ξI G) 1 Q H p > 1 }, ɛ and, when p = 2, L 2 ɛ (A) = { ξ C : (ξi G) 1 2 > 1 }, ɛ First reduce to (quasi-)triangular/hessenberg form, then, (Restarted) Lanczos/Arnoldi for each (ξi G) 1 2, or Blocked Hager [Higham/Tisseur-2000] for each Q(ξI G) 1 Q H 1. Note that this algorithm makes less sense when (ξi A) 1 can already be applied in quadratic time (e.g., if A is 3D FEM discretization) 29 / 80

30 (Extended) Van Loan algorithm [Van Loan-1985,Lui-1997] Given a reduction to condensed form, A = QGQ H, { L p ɛ (A) = ξ C : Q(ξI G) 1 Q H p > 1 }, ɛ and, when p = 2, L 2 ɛ (A) = { ξ C : (ξi G) 1 2 > 1 }, ɛ First reduce to (quasi-)triangular/hessenberg form, then, (Restarted) Lanczos/Arnoldi for each (ξi G) 1 2, or Blocked Hager [Higham/Tisseur-2000] for each Q(ξI G) 1 Q H 1. Note that this algorithm makes less sense when (ξi A) 1 can already be applied in quadratic time (e.g., if A is 3D FEM discretization) 30 / 80

31 (Extended) Van Loan algorithm for L 2 ɛ (A) 31 / 80 Algorithm: Two-norm pseudospectra via extended Van Loan algorithm Input: A F n n, shifts Ω C, restart size k Output: {φ(ξ)} ξ Ω { (ξi A) 1 2 } ξ Ω G := Schur(A), RealSchur(A), or Hessenberg(A) foreach ξ Ω do // Estimate (ξi G) 1 2 via Restarted Arnoldi Choose v 0 C n with v 0 2 = 1 while not converged do for j = 0,..., k 1 do // (ξi G) H (ξi G) 1 V j = V j H j + v j (β j e j ) H x j := (ξi G) H (ξi G) 1 v j Expand Arnoldi decomposition using x j [λ, v 0 ] := MaxEig(H k ) φ(ξ) := RealPart(λ)

32 [Hager-1984,Higham-1988] approach for L 1 ɛ (A) 32 / 80 Algorithm: One-norm pseudospectra via Hager-Higham algorithm Input: A F n n, Ω C Output: {φ(ξ)} ξ Ω { (A ξi) 1 1 } ξ Ω [Q, G] := Schur(A), RealSchur(A), or Hessenberg(A) foreach ξ Ω do // Estimate (A ξi) 1 1 via Hager-Higham algorithm Choose u C n with u 1 = 1 k := 0 repeat x := u y := Q(G ξi) 1 Q H x w := Q(G ξi) H Q H sign(y) u := e j where w(j) = w k := k + 1 until (k 2 and w w H x) or k 5 φ(ξ) := y 1 // Take the maximum between the current estimate and a heuristic b := 2 3n [( 1)j (n + j 1)] j=0:n 1 x := Q(G ξi) 1 Q H b φ(ξ) := max(φ(ξ), x 1 ) Block algorithm [Higham/Tisseur-2000] should be used in practice

33 [Hager-1984,Higham-1988] approach for L 1 ɛ (A) 33 / 80 Algorithm: One-norm pseudospectra via Hager-Higham algorithm Input: A F n n, Ω C Output: {φ(ξ)} ξ Ω { (A ξi) 1 1 } ξ Ω [Q, G] := Schur(A), RealSchur(A), or Hessenberg(A) foreach ξ Ω do // Estimate (A ξi) 1 1 via Hager-Higham algorithm Choose u C n with u 1 = 1 k := 0 repeat x := u y := Q(G ξi) 1 Q H x w := Q(G ξi) H Q H sign(y) u := e j where w(j) = w k := k + 1 until (k 2 and w w H x) or k 5 φ(ξ) := y 1 // Take the maximum between the current estimate and a heuristic b := 2 3n [( 1)j (n + j 1)] j=0:n 1 x := Q(G ξi) 1 Q H b φ(ξ) := max(φ(ξ), x 1 ) Block algorithm [Higham/Tisseur-2000] should be used in practice

34 Outline 34 / 80 Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

35 35 / 80 (Generalized) multishift solves Computing {(δ j F G) 1 y j } j equivalent to solving for X in where D = diag((δ 0, δ 1, )). FXD GX = Y, Multishift triangular solves (G triangular, F = I) require trivial changes to usual high-performance TRSM algorithms [Henry-1994] Quasi-triangular similar; adjust blocksizes to not split 2 2 Very recently used by [Gates/Haidar/Dongarra-2014] for eigenvectors of nonsymmetric matrices Generalized multishift (quasi-)triangular solves can similarly be made high-performance with a simple trick... Multishift Hessenberg solves heavily investigated for control theory [Datta et al-1994,henry-1994] Large fraction of work in Hessenberg case is level 1...

36 36 / 80 (Generalized) multishift solves Computing {(δ j F G) 1 y j } j equivalent to solving for X in where D = diag((δ 0, δ 1, )). FXD GX = Y, Multishift triangular solves (G triangular, F = I) require trivial changes to usual high-performance TRSM algorithms [Henry-1994] Quasi-triangular similar; adjust blocksizes to not split 2 2 Very recently used by [Gates/Haidar/Dongarra-2014] for eigenvectors of nonsymmetric matrices Generalized multishift (quasi-)triangular solves can similarly be made high-performance with a simple trick... Multishift Hessenberg solves heavily investigated for control theory [Datta et al-1994,henry-1994] Large fraction of work in Hessenberg case is level 1...

37 37 / 80 (Generalized) multishift solves Computing {(δ j F G) 1 y j } j equivalent to solving for X in where D = diag((δ 0, δ 1, )). FXD GX = Y, Multishift triangular solves (G triangular, F = I) require trivial changes to usual high-performance TRSM algorithms [Henry-1994] Quasi-triangular similar; adjust blocksizes to not split 2 2 Very recently used by [Gates/Haidar/Dongarra-2014] for eigenvectors of nonsymmetric matrices Generalized multishift (quasi-)triangular solves can similarly be made high-performance with a simple trick... Multishift Hessenberg solves heavily investigated for control theory [Datta et al-1994,henry-1994] Large fraction of work in Hessenberg case is level 1...

38 38 / 80 (Generalized) multishift solves Computing {(δ j F G) 1 y j } j equivalent to solving for X in where D = diag((δ 0, δ 1, )). FXD GX = Y, Multishift triangular solves (G triangular, F = I) require trivial changes to usual high-performance TRSM algorithms [Henry-1994] Quasi-triangular similar; adjust blocksizes to not split 2 2 Very recently used by [Gates/Haidar/Dongarra-2014] for eigenvectors of nonsymmetric matrices Generalized multishift (quasi-)triangular solves can similarly be made high-performance with a simple trick... Multishift Hessenberg solves heavily investigated for control theory [Datta et al-1994,henry-1994] Large fraction of work in Hessenberg case is level 1...

39 39 / 80 (Generalized) multishift solves Computing {(δ j F G) 1 y j } j equivalent to solving for X in where D = diag((δ 0, δ 1, )). FXD GX = Y, Multishift triangular solves (G triangular, F = I) require trivial changes to usual high-performance TRSM algorithms [Henry-1994] Quasi-triangular similar; adjust blocksizes to not split 2 2 Very recently used by [Gates/Haidar/Dongarra-2014] for eigenvectors of nonsymmetric matrices Generalized multishift (quasi-)triangular solves can similarly be made high-performance with a simple trick... Multishift Hessenberg solves heavily investigated for control theory [Datta et al-1994,henry-1994] Large fraction of work in Hessenberg case is level 1...

40 40 / 80 (Generalized) multishift solves Computing {(δ j F G) 1 y j } j equivalent to solving for X in where D = diag((δ 0, δ 1, )). FXD GX = Y, Multishift triangular solves (G triangular, F = I) require trivial changes to usual high-performance TRSM algorithms [Henry-1994] Quasi-triangular similar; adjust blocksizes to not split 2 2 Very recently used by [Gates/Haidar/Dongarra-2014] for eigenvectors of nonsymmetric matrices Generalized multishift (quasi-)triangular solves can similarly be made high-performance with a simple trick... Multishift Hessenberg solves heavily investigated for control theory [Datta et al-1994,henry-1994] Large fraction of work in Hessenberg case is level 1...

41 41 / 80 (Generalized) multishift solves Computing {(δ j F G) 1 y j } j equivalent to solving for X in where D = diag((δ 0, δ 1, )). FXD GX = Y, Multishift triangular solves (G triangular, F = I) require trivial changes to usual high-performance TRSM algorithms [Henry-1994] Quasi-triangular similar; adjust blocksizes to not split 2 2 Very recently used by [Gates/Haidar/Dongarra-2014] for eigenvectors of nonsymmetric matrices Generalized multishift (quasi-)triangular solves can similarly be made high-performance with a simple trick... Multishift Hessenberg solves heavily investigated for control theory [Datta et al-1994,henry-1994] Large fraction of work in Hessenberg case is level 1...

42 Interleaved Van Loan algorithm for L 2 ɛ (A) 42 / 80 Algorithm: Two-norm pseudospectra via interleaved extended Van Loan algorithm Input: A F n n, shift vector z C m, restart size k Output: f [ (z(s)i A) 1 2 ]s=0:m 1 G := Schur(A), RealSchur(A), or Hessenberg(A) Initialize each column of W 0 C n m to have unit two-norm I := (0,..., m 1) while I do for j = 0,..., k 1 do // (z(s)i G) H (z(s)i G) 1 W j (t) = W j (t)h j (t) + W j (:, t)(b j (t)e j ) H, s = I(t) X := MultishiftSolve(G, z(i), W j ) X := MultishiftSolve(G H, z(i), X) Expand Arnoldi decompositions foreach s = I(t) do [λ, W 0 (:, t)] := MaxEig(H k (t)) f (s) := RealPart(λ) if converged then Delete I(t) and W 0 (:, t)

43 Interleaved Hager-Higham algorithm for L 1 ɛ (A) 43 / 80 Algorithm: One-norm pseudospectra via interleaved Hager-Higham algorithm Input: A F n n, shift vector z C m Output: f [ (z(j)i A) 1 1 ]j=0:m 1 [Q, G] := Schur(A), RealSchur(A), or Hessenberg(A) Initialize each column of U C n m to have unit one-norm I := (0,..., m 1) k := 0 while I do X := U Y := Q H X Y := MultishiftSolve(G, z(i), Y ) Y := QY W := Q H sign(y ) W := MultishiftSolve(G H, z(i), W ) W := QW k := k + 1 foreach s = I(t) do x := X(:, t), y := Y (:, t), w := W (:, t) if (k 2 and w w H x) or k 5 then f (s) = y 1 Delete I(t) and U(:, t) else U(:, t) := e j where w(j) = w

44 Interleaved Hager-Higham algorithm for L 1 ɛ (A) 44 / 80 Algorithm: One-norm pseudospectra via interleaved Hager-Higham algorithm Input: A F n n, shift vector z C m Output: f [ (z(j)i A) 1 1 ]j=0:m 1 // Take the maximum between the current estimates and a heuristic B := 2 3n [( 1)j (n + j 1)] j=0:n 1,s=0:m 1 // All columns are equal X := Q H B // All columns are equal X := MultishiftSolve(G, z, X) X := QX foreach s = 0,..., m 1 do f (s) := max(f (s), X(:, s) 1 ) Again: block algorithm from [Higham/Tisseur-2000] should be used in practice

45 Interleaved Hager-Higham algorithm for L 1 ɛ (A) 45 / 80 Algorithm: One-norm pseudospectra via interleaved Hager-Higham algorithm Input: A F n n, shift vector z C m Output: f [ (z(j)i A) 1 1 ]j=0:m 1 // Take the maximum between the current estimates and a heuristic B := 2 3n [( 1)j (n + j 1)] j=0:n 1,s=0:m 1 // All columns are equal X := Q H B // All columns are equal X := MultishiftSolve(G, z, X) X := QX foreach s = 0,..., m 1 do f (s) := max(f (s), X(:, s) 1 ) Again: block algorithm from [Higham/Tisseur-2000] should be used in practice

46 46 / 80 Batching and deflation Maximum number of simultaneous shifts constrained by memory Could bring in new shift after each deflation, but easier to break into batches In practice, only small number of iterations needed per shift...

47 47 / 80 Batching and deflation Maximum number of simultaneous shifts constrained by memory Could bring in new shift after each deflation, but easier to break into batches In practice, only small number of iterations needed per shift...

48 48 / 80 Batching and deflation Maximum number of simultaneous shifts constrained by memory Could bring in new shift after each deflation, but easier to break into batches In practice, only small number of iterations needed per shift...

49 Outline 49 / 80 Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

50 Investigating the Fox-Li matrix with python 50 / 80 import math, El n = 100 # m a t r i x s ize realres = imagres = 100 # g r i d r e s o l u t i o n # Display an instance of the Fox L i / Landau matrix A = El. D i s t M a t r i x ( El. ztag ) El. FoxLi (A, n ) El. Display (A, " Fox L i m a t r i x " ) # Display i t s s p e c t r a l p o r t r a i t p o r t r a i t = El. S p e c t r a l P o r t r a i t (A, realres, imagres ) El. EntrywiseMap ( p o r t r a i t, math. log10 ) El. Display ( p o r t r a i t, " S p e c t r a l p o r t r a i t of Fox L i matrix " ) # Display i t s s i n g u l a r values s = El.SVD(A) El. EntrywiseMap ( s, math. log10 ) El. Display ( s, " log10 ( svd (A ) ) " )

51 Investigating the Fox-Li matrix with python 51 / 80

52 Investigating the Fox-Li matrix with python 52 / 80

53 Investigating the Fox-Li matrix with python 53 / 80

54 Outline 54 / 80 Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

55 55 / 80 FoxLi(15k), Ω = ( 1.2, 1.2) 2, pixels, 10 its (Au)(x) = if /π 1 1 e if (x y)2 u(y) dy, F = 16π 30 sec/iter on 256 cores of Stampede (256 r.h.s./core, >4 TFlops)

56 56 / 80 FoxLi(15k), Ω = ( 1.2, 1.2) 2, pixels, 20 its (Au)(x) = if /π 1 1 e if (x y)2 u(y) dy, F = 16π 30 sec/iter on 256 cores of Stampede (256 r.h.s./core, >4 TFlops)

57 57 / 80 FoxLi(15k), Ω = ( 1.2, 1.2) 2, pixels, 30 its (Au)(x) = if /π 1 1 e if (x y)2 u(y) dy, F = 16π 30 sec/iter on 256 cores of Stampede (256 r.h.s./core, >4 TFlops)

58 58 / 80 FoxLi(15k), Ω = ( 1.2, 1.2) 2, pixels, 40 its (Au)(x) = if /π 1 1 e if (x y)2 u(y) dy, F = 16π 30 sec/iter on 256 cores of Stampede (256 r.h.s./core, >4 TFlops)

59 59 / 80 FoxLi(15k), Ω = ( 1.2, 1.2) 2, pixels, 50 its (Au)(x) = if /π 1 1 e if (x y)2 u(y) dy, F = 16π 30 sec/iter on 256 cores of Stampede (256 r.h.s./core, >4 TFlops)

60 FoxLi(15k) work, Ω = ( 1.2, 1.2) 2, pix, 10 its 60 / 80

61 FoxLi(15k) work, Ω = ( 1.2, 1.2) 2, pix, 20 its 61 / 80

62 FoxLi(15k) work, Ω = ( 1.2, 1.2) 2, pix, 30 its 62 / 80

63 FoxLi(15k) work, Ω = ( 1.2, 1.2) 2, pix, 40 its 63 / 80

64 FoxLi(15k) work, Ω = ( 1.2, 1.2) 2, pix, 50 its 64 / 80

65 65 / 80 Uniform(20k), Ω = ( 103, 103) 2, pix pieces: 75 sec/iter on 256 nodes of Blue Gene/Q, (64 r.h.s./core, >11 TFlops), 1175 sec for Schur

66 66 / 80 Uniform(20k), Ω = ( 10.3, 10.3) 2, pix pieces: 75 sec/iter on 256 nodes of Blue Gene/Q, (64 r.h.s./core, >11 TFlops), 1175 sec for Schur

67 Outline 67 / 80 Van Loan s algorithm and the Demmel matrix Brief overview of pseudospectra Previous work High-performance batched analogues A brief example of the python interface Results Conclusions and future work

68 68 / 80 Convergence issues Typically small number of iterations required for visual convergence. Given current batch iteration, can output image after each fixed number of iterations Problematic pixels typically uninteresting (resolvent gradient is small)

69 69 / 80 Future directions Interleaved block one-norm pseudospectral algorithm More intelligent interpolation and stopping criteria Investigate (preconditioned) 2D/3D wave equations Better understanding of practical limits of SDC EVD Optional projection onto relevant eigenspaces Complex distributed Hessenberg QR/QZ with AED... Black-box high-performance generalized pseudospectra

70 70 / 80 Future directions Interleaved block one-norm pseudospectral algorithm More intelligent interpolation and stopping criteria Investigate (preconditioned) 2D/3D wave equations Better understanding of practical limits of SDC EVD Optional projection onto relevant eigenspaces Complex distributed Hessenberg QR/QZ with AED... Black-box high-performance generalized pseudospectra

71 71 / 80 Future directions Interleaved block one-norm pseudospectral algorithm More intelligent interpolation and stopping criteria Investigate (preconditioned) 2D/3D wave equations Better understanding of practical limits of SDC EVD Optional projection onto relevant eigenspaces Complex distributed Hessenberg QR/QZ with AED... Black-box high-performance generalized pseudospectra

72 72 / 80 Future directions Interleaved block one-norm pseudospectral algorithm More intelligent interpolation and stopping criteria Investigate (preconditioned) 2D/3D wave equations Better understanding of practical limits of SDC EVD Optional projection onto relevant eigenspaces Complex distributed Hessenberg QR/QZ with AED... Black-box high-performance generalized pseudospectra

73 73 / 80 Future directions Interleaved block one-norm pseudospectral algorithm More intelligent interpolation and stopping criteria Investigate (preconditioned) 2D/3D wave equations Better understanding of practical limits of SDC EVD Optional projection onto relevant eigenspaces Complex distributed Hessenberg QR/QZ with AED... Black-box high-performance generalized pseudospectra

74 74 / 80 Future directions Interleaved block one-norm pseudospectral algorithm More intelligent interpolation and stopping criteria Investigate (preconditioned) 2D/3D wave equations Better understanding of practical limits of SDC EVD Optional projection onto relevant eigenspaces Complex distributed Hessenberg QR/QZ with AED... Black-box high-performance generalized pseudospectra

75 75 / 80 Future directions Interleaved block one-norm pseudospectral algorithm More intelligent interpolation and stopping criteria Investigate (preconditioned) 2D/3D wave equations Better understanding of practical limits of SDC EVD Optional projection onto relevant eigenspaces Complex distributed Hessenberg QR/QZ with AED... Black-box high-performance generalized pseudospectra

76 Acknowledgments and Availability 76 / 80 Support Computational resources My hosts Jakub Kurzak and Jack Dongarra Availability Elemental is available under the New BSD License at libelemental.org Questions?

77 Generalized multishift (quasi-)trsm FXD GX = Y As long as a 2x2 block is not split: ( ) (F1,1 X 1 D G 1,1 X 1 ) + (F 1,2 X 2 D G 1,2 X 2 ) = F 2,2 X 2 D G 2,2 X 2 ( Y1 If F 2,2 and G 2,2 are small and square, have each process locally solve for a few right-hand sides of X 2. Then, form Ŷ 1 := Y 1 (F 1,2 X 2 D G 1,2 X 2 ) via a parallel GEMM with each column of F 1,2 X 2 appropriately scaled afterwards. All that is left is to recurse on F 1,1 X 1 D G 1,1 X 1 = Ŷ1. Y 2 ) When F = I, algorithm is much simpler. 77 / 80

78 Generalized multishift (quasi-)trsm FXD GX = Y As long as a 2x2 block is not split: ( ) (F1,1 X 1 D G 1,1 X 1 ) + (F 1,2 X 2 D G 1,2 X 2 ) = F 2,2 X 2 D G 2,2 X 2 ( Y1 If F 2,2 and G 2,2 are small and square, have each process locally solve for a few right-hand sides of X 2. Then, form Ŷ 1 := Y 1 (F 1,2 X 2 D G 1,2 X 2 ) via a parallel GEMM with each column of F 1,2 X 2 appropriately scaled afterwards. All that is left is to recurse on F 1,1 X 1 D G 1,1 X 1 = Ŷ1. Y 2 ) When F = I, algorithm is much simpler. 78 / 80

79 Generalized multishift (quasi-)trsm FXD GX = Y As long as a 2x2 block is not split: ( ) (F1,1 X 1 D G 1,1 X 1 ) + (F 1,2 X 2 D G 1,2 X 2 ) = F 2,2 X 2 D G 2,2 X 2 ( Y1 If F 2,2 and G 2,2 are small and square, have each process locally solve for a few right-hand sides of X 2. Then, form Ŷ 1 := Y 1 (F 1,2 X 2 D G 1,2 X 2 ) via a parallel GEMM with each column of F 1,2 X 2 appropriately scaled afterwards. All that is left is to recurse on F 1,1 X 1 D G 1,1 X 1 = Ŷ1. Y 2 ) When F = I, algorithm is much simpler. 79 / 80

80 Generalized multishift (quasi-)trsm FXD GX = Y As long as a 2x2 block is not split: ( ) (F1,1 X 1 D G 1,1 X 1 ) + (F 1,2 X 2 D G 1,2 X 2 ) = F 2,2 X 2 D G 2,2 X 2 ( Y1 If F 2,2 and G 2,2 are small and square, have each process locally solve for a few right-hand sides of X 2. Then, form Ŷ 1 := Y 1 (F 1,2 X 2 D G 1,2 X 2 ) via a parallel GEMM with each column of F 1,2 X 2 appropriately scaled afterwards. All that is left is to recurse on F 1,1 X 1 D G 1,1 X 1 = Ŷ1. Y 2 ) When F = I, algorithm is much simpler. 80 / 80