GPU accelerated Arnoldi solver for small batched matrix

Transcription

1 GPU accelerated Arnoldi solver for small batched matrix Samsung Advanced Institute of Technology Hyung-Jin Kim

2 Contents - Eigen value problems - Solution - Arnoldi Algorithm - Target - CUDA optimization

3 Eigen Value Problem A Ax = λx a 00 a 0n n x n, complex : x a n0 a nn valued matrix, a 0 a n : vector(x) which satisfies above linear equation λ : scalar value(λ) which satisfies above linear equation Hψ E = Eψ E Solution of Schrödinger Equation mx + kx = 0 Vibration analysis Principal Component Analysis Eigen faces

4 How to solve eigenvalue problems? - Find a solution of following equation, - For 2x2 case, det (A λi) = 0 A a b c d, I det a λ b c d λ = a λ d λ bc = 0 λ = a + d ± (a d)2 +4bc 2 - No exact solution for finding n roots of n-th order polynomial equation - Iterative method : Power method, QR, Arnoldi, Lanczos,

5 Arnoldi Algorithm v 1 = v/ v for k = 1 to m -1 do v k+1 = Av k for j = 1 to k do h jk = v j *v k+1 v k+1 = v k+1 -v j h jk end for /*Gram-Schmidt orthogonalization*/ h k+1,k = v k+1 if h k+1,k = 0 then return {v 1,,v k } /*is invariant under A*/ end if v k+1 = v k+1 /h k+1,k end for - Arnoldi algorithm just gives k-orthonormal basis and Hessenberg matrix H - If matrix A is hermitian, iteration sequence becomes simpler form : Lanczos algorithm H a 00 a 01 a 02 a 03 a 10 a 11 a 12 a 13 0 a 21 a 22 a a 32 a 33

6 Further works (1) - QR factorization : finds upper trianglar matrix T, unitary matrix U s.t A = UTU* Set A 0 = A and U 0 = I. for k = 1, 2,..., do Q k R k = A k 1 ; /* QR factorization */ A k = R k Q k ; U k = U k-1 Q k ; /* Update transformation matrix */ end for Set T = A and U = U. - In general, QR factorization is O(3) algorithm computationally expensive! - QR rotation : define a rotation transform G(i,j,θ), G i, j, θ = H G 0,1, θ 0 G,, θ = R G 1,2, θ 1 : O(2) algorithm! Computationally cheap! c = c s 0 0 s c x i x i 2 + x j 2 i s = j x j i j x i 2 + x j 2

7 Further works (2) - Diagonal elements of T are eigenvalues of A : Done! (Eigenvalues are preserved under similarity transform) - From H λ i I s i = 0, we can calculate n-unknown elements of i-th eigenvector of Hessenberg matrix - Eigenvector of A can be derived from x i = Vs i, V v 1 v 2 v n - Overally, most of the computations are O(2) algorithms except x i computation

8 Implementation strategy - Matrix size is less than 1000x Complex, non-symmetric matrix - parallel process for more than ~100 of matrices simultaneous kernel run - Each matrices are aligned sequentially in memory batched data - CUBLAS library(ver >7.0) is good enough in most of BLAS computations - Arnoldi, QR algorithm structural sub-routine calls are executed from CPU side - cudastream is good solution for parallel(or simultaneous) kernel launching Input data a 00 a 0n a n0 a nn k+1 a 00 a 0n a n0 a nn k Current Solver GPU kernel output data d d n k 1 d d n k 2 Input data a 00 a 0n a 00 a 0n a a 00 a n0 a 0n a nn k+1 a 00 a n0 a 0n nn k a a 00 a n0 a 0n a nn k+1 a 00 a n0 a 0n nn k a a 00 a n0 a 0n a nn k+1 a 00 a n0 a 0n nn k a n0 a nn k+1 a n0 a nn k cudastream threaded Solver Kernel 0 Kernel 1 Kernel 2 Kernel 3 output data d 0 0 d 0 0 d d d n d k 1 n k 2 d d d n d k 1 n k 2 d d d n d k 1 n k 2 0 d n 0 d k 1 n k 2

9 Implementation Detail cudastreamcreate(&stream1); cudastreamcreate(&stream2); kernel1<<<grid, block, 0, stream1>>>(data_1); kernel2<<<grid, block, 0, stream2>>>(data_2); Nsight Profiler view 8 concurrent kernel launch - CUBLAS also supports cudastream with other name, cudastreamcreate() and cublassetstream() - For <t>gemm(), <t>trsm() operations, batch mode is natively supported

10 Performance(1) Tested on Xeon E5-2680v2, K80 GPUs - Dgemm operation MKL(GF) Single kernel(gf) 10 stream kernels(gf) 10/Single ratio - Dgemv operation 128x x MKL(GF) Single kernel(gf) 10 stream kernels(gf) 10/Single ratio 128x x Dgemv performance on different kernel size Single kernel (GF) 10 kernels (GF) 100 kernels (GF) 1000 kernels (GF) 128x x

11 Performance(2) Tested on Xeon E5-2680v2, K80 GPUs - Full eigenvalue evaluation sequences are still under developing - Tested Arnoldi iterations only, 10 matrix solve (preliminary!) - Intel MKL, MAGMA library data are given MKL(sec) Magma(sec) Optimized solver(sec) 256x x

12 Potential problems? - Maximum number of cudastream is unknown(help!) - Fermi GPU : 16, Kepler GPU : 32 concurrent kernel run - Maxwell GPU : Unknown(Help!) If matrix size is too small, GPU utilization could be still low per-thread option would be useful? - Shaded 4 elements are not continually aligned on GPU memory - For QR rotation, we don t need to fully access on Hessenberg matrix - Custom data structure for Hessenberg matrix might be useful! - Eigenvectors are calculated from H λ i I s i = 0 - Eigenvector elements are calculated in sequential way - concurrent eigenvector computation is also needed Backward computation s i0 s i1 s i2 s i3 = 0 43 s i2 = - 44 s i3,

13 Thank you! Ref) Arnoldi Algorithm : Kernel Streaming : CUDA_C_Programming_Guide, CUDA SDK(concurrentKernels) CUBLAS : CUBLAS_Library, CUDA SDK(batchCUBLAS)