An Empirical Comparison of Graph Laplacian Solvers

Transcription

1 An Empirical Comparison of Graph Laplacian Solvers Kevin Deweese 1 Erik Boman 2 John Gilbert 1 1 Department of Computer Science University of California, Santa Barbara 2 Scalable Algorithms Department Sandia National Laboratories Algorithm Engineering and Experiments, 2016 Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National Nuclear Security Administration under contract DE-AC04-94AL85000.

2 Problem Description Our focus: Solve the the system of equations Lx = b where L is a graph Laplacian matrix K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

3 Applications Graphs with regular degree structure, 2D/3D meshes Finite element analysis Electrical and thermal conductivity Fluid flow modeling Image processing Image segmentation, inpainting, regression, classification Graphs with irregular degree, problems in network analysis Maximum flow problems Graph sparsification Spectral clustering K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

4 Applications Graphs with regular degree structure, 2D/3D meshes Finite element analysis Electrical and thermal conductivity Fluid flow modeling Image processing Image segmentation, inpainting, regression, classification Graphs with irregular degree, problems in network analysis Maximum flow problems Graph sparsification Spectral clustering K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

5 SDD Systems Some applications use symmetric diagonally dominant (SDD) matrices Slightly more general, allows for positive off-diagonal entries Can be reduced to solving a Laplacian linear system K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

6 SDD Solvers With Good Asymptotic Complexity Linear times polylog. Spielman and Teng, 2006 Nearly m log n. Koutis, Miller, and Peng, 2011 A simple, combinatorial algorithm. Kelner, Orecchia, Sidford, and Zhu, 2013 mostly theoretical results, few experiments K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

7 Our Goal Perform a comprehensive study of existing Laplacian solvers Select a set of test problems that are relevant/challenging Select performance metrics for evaluating current and future Laplacian solver performance Ongoing work, plan to update it with new solvers, new test problems K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

8 Test Graphs University of Florida Sparse Matrix Collection [Davis] Irregular degree graphs - 10K edges to 4M edges 2D/3D mesh-like graphs - 30K edges to 7M edges Block two-level Erdös Rényi (BTER) [Seshadhri et al.] Designed to model web graphs with realistic degree distributions and clustering behavior Image segmentation graphs [Felzenszwalb and Huttenlocher] Pixels=vertices, edge weights represent dissimilarity between pixel values - 300K edges to 7M edges K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

9 Solution Methods Direct methods Solve in a finite number of operations Cholesky factorization Sometimes expensive in time and memory use Iterative methods Form a sequence of improving approximations Conjugate gradients (CG) Convergence depends on matrix spectrum, bounded in terms of condition number κ(a) = λ n(a)/λ 1 (A) Typically used with a preconditioner to improve the condition number Multilevel Approximate solution on a coarser problem, occasionally correct on original Form recursive hierarchy of approximations K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

10 Solution Methods all from [Trilinos] Direct Solvers Cholesky factorization (Cholmod [Davis]) CG with single-level preconditioner Jacobi Incomplete LU Factorization (ILU) Spanning trees CG with multi-level preconditioner Algebraic Multigrid (AMG) K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

11 Solution Methods all from [Trilinos] Direct Solvers Cholesky factorization (Cholmod [Davis]) CG with single-level preconditioner Jacobi Incomplete LU Factorization (ILU) Spanning trees CG with multi-level preconditioner Algebraic Multigrid (AMG) = x K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

12 Solution Methods all from [Trilinos] Direct Solvers Cholesky factorization (Cholmod [Davis]) CG with single-level preconditioner Jacobi Incomplete LU Factorization (ILU) Spanning trees CG with multi-level preconditioner Algebraic Multigrid (AMG) D K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

13 Solution Methods all from [Trilinos] Direct Solvers Cholesky factorization (Cholmod [Davis]) CG with single-level preconditioner Jacobi Incomplete LU Factorization (ILU) Spanning trees CG with multi-level preconditioner Algebraic Multigrid (AMG) x K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

14 Solution Methods all from [Trilinos] Direct Solvers Cholesky factorization (Cholmod [Davis]) CG with single-level preconditioner Jacobi Incomplete LU Factorization (ILU) Spanning trees CG with multi-level preconditioner Algebraic Multigrid (AMG) T = x K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

15 Solution Methods all from [Trilinos] Direct Solvers Cholesky factorization (Cholmod [Davis]) CG with single-level preconditioner Jacobi Incomplete LU Factorization (ILU) Spanning trees CG with multi-level preconditioner Algebraic Multigrid (AMG) K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

16 Experimental Design Lx = b solved on problems in 4 test sets b randomly generated Solutions found to within residual tolerance of 10 9 Mostly used default solver parameters K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

17 Performance Metrics Number of Iterations Setup Time (one time work) Per-solve Time (every time work) Total Time (Setup+Per-solve) Memory Usage K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

18 Setup + Per-solve (Irregular) Fraction of problems within τ of best Fraction of problems within τ of best Cholesky Jacobi ILU Tree Multilevel τ τ UF Irregular Graphs BTER Graphs K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

19 Setup + Per-solve (Mesh-like) Fraction of problems within τ of best Fraction of problems within τ of best Cholesky Jacobi ILU Tree Multilevel τ τ UF Mesh-like Graphs Image Segmentation Graphs K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

20 Iterations (Irregular) Iterations 10 2 Iterations 10 2 Jacobi ILU Tree Multilevel κ(l) UF Irregular Graphs κ(l) BTER Graphs K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

21 Iterations (Mesh-like) Iterations 10 2 Iterations 10 2 Jacobi ILU Tree Multilevel κ(l) UF Mesh-like Graphs κ(l) Image Segmentation Graphs K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

22 Per-solve Time (Irregular) Per-Solve Time(s) Per-Solve Time(s) Cholesky Jacobi ILU Tree Multilevel m κ(l) UF Irregular Graphs m κ(l) BTER Graphs K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

23 Per-solve Time (Mesh-like) Per-Solve Time(s) Per-Solve Time(s) Cholesky Jacobi ILU Tree Multilevel m κ(l) UF Mesh-like Graphs m κ(l) Image Segmentation Graphs K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

24 Summary of Results Relative solver performance is consistent within test sets, but very different between test sets Multigrid does well on mesh-like problems; single-level preconditioners do well on irregular problems BTER problems are easier for the iterative methods, more difficult for direct methods The irregular problems are better conditioned, simple preconditioners like Jacobi do well K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

25 Future Work Incorporate additional solvers Multigrid methods designed for irregular graphs Decide how to incorporate solvers outside Trilinos Add additional test problems Understand how graph structure -> condition number, solver behavior K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20

26 Thank You K. Deweese, J. Gilbert, E. Boman An Empirical Comparison of Graph Laplacian Solvers ALENEX / 20