Sparse Matrices and Iterative Methods

Transcription

1 Sparse Matrices and Iterative Methods K. 1 1 Department of Mathematics 2018

2 Iterative Methods Consider the problem of solving Ax = b, where A is n n. Why would we use an iterative method? Avoid direct decomposition (LU, QR, Cholesky) Replace with iterated matrix multiplication LU is O(n 3 ) flops matrix-vector multiplication is O(n 2 )... so if we can get convergence in e.g. log(n), iteration might be faster.

3 Jacobi, GS, SOR Some old methods: Jacobi is easily parallelized but converges extremely slowly. Gauss-Seidel/SOR converge faster but cannot be effectively parallelized. Only Jacobi really takes advantage of sparsity.

4 When a matrix is sparse (many more zero entries than nonzero), then typically the number of nonzero entries is O(n), so matrix-vector multiplication becomes an O(n) operation. This makes iterative methods very attractive. It does not help direct solves as much because of the problem of fill-in, but we note that there are specialized solvers to minimize fill-in.

5 Krylov Subspace Methods A class of methods that converge in n iterations (in exact arithmetic). We hope that they arrive at a solution that is close enough in fewer iterations. Often these work much better than the classic methods. They are more readily parallelized, and take full advantage of sparsity.

6 Possibilities Sparse matrices are quite common in computation Finite differences for PDEs Finite element for PDEs Integral equations with localized kernels

7 Structures Nonzero elements shown in blue. Note: nz denotes number of nonzeros out of 17 million entries.

8 Some formats There are a few ways to store sparse matrices that are obvious. Diagonals - 1+ entry per nonzero Coordinates - 3 entries per nonzero Row- or column-oriented coordinates - 2+ entries per nonzero

9 Diagonal (DIA) Matrix Sparse Storage Offsets: [ ]

10 Coordinate (COO) Matrix Sparse Storage [ ] [ ] [ ] rows cols vals Often used for conversions...

11 Compressed Sparse Row (CSR) Matrix Sparse Storage [ ] [ ] [ ] row cols vals offsets Also Compressed Sparse Column format, for multiplications.

12 Sparse Package Scipy has a subpackage called sparse that implements many of these formats. Diagonal: dia_matrix() Coordinate: coo_matrix() CSR, CSC: csr_matrix(), csc_matrix()... and others.

13 Using Sparse from scipy import * from scipy.sparse import csr_matrix A = csr_matrix([[-1,1,0,0],[0,-2,0,0],[0,-3,0,5],\ [0,0,1,1]]) x = array([1, 0, -1,0]) y = A.dot(x) print(y) print(a) Results in y=[-1,0,0,-1]. A prints in COO format. Note that dot(a,x) does not work. dot must be the method of the sparse object.

14 Example As a very simple example of the efficacy of the sparse matrix package in scipy, consider the PDE x = 1, x Ω = 0, where the region Ω is the unit square. We solve this numerically using finite differences.

15 Matrix There are many ways to assemble the matrix. Here is one. N = 100 Nsq = N*N h = 1.0/float(N+1) offsets = [-N,-1,0,1,N] subdiag1 = ones(nsq) subdiag1[n-1:nsq:n] = 0. supdiag1 = ones(nsq) supdiag1[0:nsq:n] = 0. A = dia_matrix(([-ones(nsq),-subdiag1,4.*ones(nsq),\ -supdiag1,-ones(nsq)],offsets),shape=(nsq,nsq))

16 Conversion It is easy to convert to other formats: e.g. given our DIA format matrix A, we can get other formats using: Acsr = A.tocsr() Afull = A.toarray()

17 Solve We can solve the systems using various methods. Given: from scipy.linalg import solve as lsolve import scipy.sparse.linalg as sp Sparse Conjugate Gradient: soln = sp.cg(a,h*h*ones(nsq)) Sparse LU: solnsp = sp.spsolve(acsr,h*h*ones(nsq)) Full LU: solnfull = lsolve(afull,h*h*ones(nsq))

18 Results... for a system of interior finite difference points (i.e matrix) Sparse CG: seconds, 2.7e-7 difference from full Sparse LU: seconds, 1.9e-13 difference from full Full LU: 15.2 seconds, 0 difference Of course, for a more serious problem we would precondition etc.