Evaluation and Benchmarking of Highly Scalable Parallel Numerical Libraries
|
|
|
- George Burke
- 5 years ago
- Views:
Transcription
1 Evaluation and Benchmarking of Highly Scalable Parallel Numerical Libraries Christos Theodosiou User and Application Support Scientific Computing AUTH
2 Presentation Outline Problem Description Serial Implementation Parallel implementation Numerical Results Conclusions 2
3 Problem Description 3
4 Linear Algebra Linear System Solution (LAPACK/SCALAPACK) Ax=b Matrix-Matrix Multiplication (BLAS/PBLAS) Ax-b=0 4
5 Linear Algebra Libraries Serial Implementation BLAS (Basic Linear Algebra Subprograms) LAPACK (Linear Algebra PACKage) Parallel Implementation BLACS (Basic Linear Algebra Communication Subprograms) PBLAS (Parallel BLAS) SCALAPACK (Scalable LAPACK) 5
6 Example Case 6
7 Example Case N M 7
8 Example Case N N RHS M 8
9 Serial Implementation (Ax=b) DGESV: Computes the solution to a real system of linear equations A*X=B DGESV( N, NRHS, A, LDA, IPIV, B, LDB, INFO ) N : The number of linear equations. NRHS: The number of right hand sides. A: On entry, the N-by-N matrix A. On exit, the factors L and U of A. LDA: The leading dimension of the matrix A. IPIV: The pivot indices that define the permutation matrix P B: On entry, the matrix of right hand side matrix B. On exit, the N-by-NRHS solution matrix X. LDB: The leading dimension of the array B. INFO: If equal to zero the solve was successful. 9
10 Serial Implementation (Ax-b=0) DGEMM: Perform one of the matrix-matrix operations C = a*a*b + b*c DGEMM( TRANSA, TRANSB, M, N, K, ALPHA, A, LDA, B, LDB, BETA, C, LDC ) TRANSA: Specifies if normal or transpose matrix A will be used. TRANSB: Specifies if normal or transpose matrix B will be used. M: The number of rows of the matrix A and C. N: The number of columns of the matrix B and C. K: The number of columns of the matrix A and rows of the matrix B. ALPHA: The scalar alpha. A: The M-by-K matrix A. LDA: The leading dimension of the matrix A. B: The K-by-N matrix B. LDB: The leading dimension of the matrix A. BETA: The scalar beta. C: The M-by-N matrix C. LDC: The leading dimension of the matrix A. 10
11 Example Case (Parallel Implementation) 11
12 Example Case (Parallel Implementation) N N RHS M 12
13 Example Case (Parallel Implementation) 2 x 2 = 4 cpus N N RHS M 13
14 Example Case (Parallel Implementation) 3 x 2 = 6 cpus N N RHS M 14
15 Parallel Implementation (Ax=b) PDGESV: Computes the solution to a real system of linear equations A*X=B DGESV ( N, NRHS, A, LDA, IPIV, B, LDB, INFO ) PDGESV( N, NRHS, A, IA, JA, DESCA, IPIV, B, IB, JB, DESCB, INFO ) IA: The row index in the global array A. JA: The column index in the global array A. DESCA: The array descriptor for the distributed matrix A. IB: The row index in the global array B. JB: The column index in the global array B. DESCB: The array descriptor for the distributed matrix B. 15
16 Parallel Implementation (Ax-b=0) PDGEMM: Perform one of the matrix-matrix operations C = a*a*b + b*c DGEMM ( TRANSA, TRANSB, M, N, K, ALPHA, A, LDA, B, LDB, BETA, C, LDC ) PDGEMM( TRANSA, TRANSB, M, N, K, ALPHA, A, IA, JA, DESCA, B, IB, JB, DESCB, BETA, C, IC, JC, DESCC ) IA: The row index in the global array A. JA: The column index in the global array A. DESCA: The array descriptor for the distributed matrix A. IB: The row index in the global array B. JB: The column index in the global array B. DESCB: The array descriptor for the distributed matrix B. IC: The row index in the global array C. JC: The column index in the global array C. DESCC: The array descriptor for the distributed matrix C. 16
17 Serial Implementation Standard BLAS Goto BLAS ATLAS BLAS ACML (AMD Core Math Library) Intel MKL 17
18 Serial Implementation Results (Ax-b=0) 00:35 00:30 00:26 00:22 Time 00:17 00:13 00:09 00:04 blas goto 1p goto 8p 00: Size * Intel Xeon 2.33GHz 18
19 Serial Implementation Results (Ax=b) 14:00 12:00 10:00 Time 08:00 06:00 04:00 02:00 blas goto 1p goto 8p 00: Size * Intel Xeon 2.33GHz 19
20 Parallel Implementation Results (Ax-b=0) Time 00:43 00:39 00:35 00:30 00:26 00:22 00:17 00:13 00:09 00:04 00: Size blas goto 1p goto 8p scalapack 1x8 scalapack 8x8 * Intel Xeon 2.33GHz 20
21 Parallel Implementation Results (Ax=b) 14:00 01:00 12:00 Time 10:00 00:40 08:00 06:00 00:20 04:00 02:00 blas goto 1p scalapack 1x8 goto 8p scalapack 8x8 scalapack 1x8 scalapack 8x8 00: Size * Intel Xeon 2.33GHz 21
22 Conclusions Optimized Linear Algebra Libraries improve performance Scale becomes better as the problems get bigger Distributed Memory Libraries can treat larger problems than Shared Memory Libraries 22
Symmetric Pivoting in ScaLAPACK Craig Lucas University of Manchester Cray User Group 8 May 2006, Lugano
Symmetric Pivoting in ScaLAPACK Craig Lucas University of Manchester Cray User Group 8 May 2006, Lugano Introduction Introduction We wanted to parallelize a serial algorithm for the pivoted Cholesky factorization
Intel Math Kernel Library (Intel MKL) LAPACK
Intel Math Kernel Library (Intel MKL) LAPACK Linear equations Victor Kostin Intel MKL Dense Solvers team manager LAPACK http://www.netlib.org/lapack Systems of Linear Equations Linear Least Squares Eigenvalue
ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance (Technical Paper)
ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance (Technical Paper) L. S. Blackford y, J. Choi z, A. Cleary x,j.demmel {, I. Dhillon{, J. Dongarra
Some notes on efficient computing and setting up high performance computing environments
Some notes on efficient computing and setting up high performance computing environments Andrew O. Finley Department of Forestry, Michigan State University, Lansing, Michigan. April 17, 2017 1 Efficient
Binding Performance and Power of Dense Linear Algebra Operations
10th IEEE International Symposium on Parallel and Distributed Processing with Applications Binding Performance and Power of Dense Linear Algebra Operations Maria Barreda, Manuel F. Dolz, Rafael Mayo, Enrique
Parallelization of the Molecular Orbital Program MOS-F
Parallelization of the Molecular Orbital Program MOS-F Akira Asato, Satoshi Onodera, Yoshie Inada, Elena Akhmatskaya, Ross Nobes, Azuma Matsuura, Atsuya Takahashi November 2003 Fujitsu Laboratories of
routines for distributed memory MIMD computers. It is a continuation fast as possibleè, scalability èas the problem size and number of processors
ScaLAPACK Tutorial? Jack Dongarra 1;2 and L. Susan Blackford??1 1 Department of Computer Science, University oftennessee, Knoxville, TN 37996-1301, USA 2 Mathematical Sciences Section, Oak Ridge National
INITIAL INTEGRATION AND EVALUATION
INITIAL INTEGRATION AND EVALUATION OF SLATE PARALLEL BLAS IN LATTE Marc Cawkwell, Danny Perez, Arthur Voter Asim YarKhan, Gerald Ragghianti, Jack Dongarra, Introduction The aim of the joint milestone STMS10-52
( ) ? ( ) ? ? ? l RB-H
1 2018 4 17 2 30 4 17 17 8 1 1. 4 10 ( ) 2. 4 17 l 3. 4 24 l 4. 5 1 l ( ) 5. 5 8 l 2 6. 5 15 l - 2018 8 6 24 7. 5 22 l 8. 6 5 l - 9. 6 12 l 10. 6 19? l l 11. 7 3? l 12. 7 10? l 13. 7 17? l RB-H 3 4 5 6
Accelerating Linear Algebra on Heterogeneous Architectures of Multicore and GPUs using MAGMA and DPLASMA and StarPU Schedulers
UT College of Engineering Tutorial Accelerating Linear Algebra on Heterogeneous Architectures of Multicore and GPUs using MAGMA and DPLASMA and StarPU Schedulers Stan Tomov 1, George Bosilca 1, and Cédric
ECS289: Scalable Machine Learning
ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Sept 27, 2015 Outline Linear regression Ridge regression and Lasso Time complexity (closed form solution) Iterative Solvers Regression Input: training
2018/10/
1 2018 10 2 2 30 10 2 17 8 1 1. 9 25 2. 10 2 l 3. 10 9 l 4. 10 16 l 5. 10 23 l 2 6. 10 30 l - 2019 2 4 24 7. 11 6 l 8. 11 20 l - 9. 11 27 l 10. 12 4 l l 11. 12 11 l 12. 12 18?? l 13. 1 8 l RB-H 3 4 5 6
SPARSE SOLVERS POISSON EQUATION. Margreet Nool. November 9, 2015 FOR THE. CWI, Multiscale Dynamics
SPARSE SOLVERS FOR THE POISSON EQUATION Margreet Nool CWI, Multiscale Dynamics November 9, 2015 OUTLINE OF THIS TALK 1 FISHPACK, LAPACK, PARDISO 2 SYSTEM OVERVIEW OF CARTESIUS 3 POISSON EQUATION 4 SOLVERS
F08BEF (SGEQPF/DGEQPF) NAG Fortran Library Routine Document
NAG Fortran Library Routine Document Note. Before using this routine, please read the Users Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent
COMPUTATIONAL METHODS IN MRI: MATHEMATICS
COMPUTATIONAL METHODS IN MATHEMATICS Imaging Sciences-KCL November 20, 2008 OUTLINE 1 MATRICES AND LINEAR TRANSFORMS: FORWARD OUTLINE 1 MATRICES AND LINEAR TRANSFORMS: FORWARD 2 LINEAR SYSTEMS: INVERSE
Roundoff Error. Monday, August 29, 11
Roundoff Error A round-off error (rounding error), is the difference between the calculated approximation of a number and its exact mathematical value. Numerical analysis specifically tries to estimate
Aasen s Symmetric Indefinite Linear Solvers in LAPACK
Aasen s Symmetric Indefinite Linear Solvers in LAPACK Ichitaro Yamazaki and Jack Dongarra December 2, 217 Abstract Recently, we released two LAPACK subroutines that implement Aasen s algorithms for solving
Matrix Eigensystem Tutorial For Parallel Computation
Matrix Eigensystem Tutorial For Parallel Computation High Performance Computing Center (HPC) http://www.hpc.unm.edu 5/21/2003 1 Topic Outline Slide Main purpose of this tutorial 5 The assumptions made
Accelerating linear algebra computations with hybrid GPU-multicore systems.
Accelerating linear algebra computations with hybrid GPU-multicore systems. Marc Baboulin INRIA/Université Paris-Sud joint work with Jack Dongarra (University of Tennessee and Oak Ridge National Laboratory)
Note Set 3 Numerical Linear Algebra
Note Set 3 Numerical Linear Algebra 3.1 Overview Numerical linear algebra includes a class of problems which involves multiplying vectors and matrices, solving linear equations, computing the inverse of
Matrix operations Linear Algebra with Computer Science Application
Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the
MATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.
MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Row-column rule: i-j-th entry of AB:
A distributed packed storage for large dense parallel in-core calculations
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. 2007; 19:483 502 Published online 28 September 2006 in Wiley InterScience (www.interscience.wiley.com)..1119 A
Tensor Contractions with Extended BLAS Kernels on CPU and GPU
Tensor Contractions with Extended BLAS Kernels on CPU and GPU Cris Cecka Senior Research Scientist NVIDIA Research, Santa Clara, California Joint work with Yang Shi, U.N. Niranjan, and Animashree Anandkumar
Communication-avoiding LU and QR factorizations for multicore architectures
Communication-avoiding LU and QR factorizations for multicore architectures DONFACK Simplice INRIA Saclay Joint work with Laura Grigori INRIA Saclay Alok Kumar Gupta BCCS,Norway-5075 16th April 2010 Communication-avoiding
Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4
Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix
NAG Library Routine Document F08QVF (ZTRSYL)
F8 Least-squares and Eigenvalue Problems (LAPAK) F8QVF NAG Library Routine Document F8QVF (ZTRSYL) Note: before using this routine, please read the Users Note for your implementation to check the interpretation
Porting a Sphere Optimization Program from lapack to scalapack
Porting a Sphere Optimization Program from lapack to scalapack Paul C. Leopardi Robert S. Womersley 12 October 2008 Abstract The sphere optimization program sphopt was originally written as a sequential
Re-design of Higher level Matrix Algorithms for Multicore and Heterogeneous Architectures. Based on the presentation at UC Berkeley, October 7, 2009
III.1 Re-design of Higher level Matrix Algorithms for Multicore and Heterogeneous Architectures Based on the presentation at UC Berkeley, October 7, 2009 Background and motivation Running time of an algorithm
A model leading to self-consistent iteration computation with need for HP LA (e.g, diagonalization and orthogonalization)
A model leading to self-consistent iteration computation with need for HP LA (e.g, diagonalization and orthogonalization) Schodinger equation: Hψ = Eψ Choose a basis set of wave functions Two cases: Orthonormal
Solutions of Linear system, vector and matrix equation
Goals: Solutions of Linear system, vector and matrix equation Solutions of linear system. Vectors, vector equation. Matrix equation. Math 112, Week 2 Suggested Textbook Readings: Sections 1.3, 1.4, 1.5
S N. hochdimensionaler Lyapunov- und Sylvestergleichungen. Peter Benner. Mathematik in Industrie und Technik Fakultät für Mathematik TU Chemnitz
Ansätze zur numerischen Lösung hochdimensionaler Lyapunov- und Sylvestergleichungen Peter Benner Mathematik in Industrie und Technik Fakultät für Mathematik TU Chemnitz S N SIMULATION www.tu-chemnitz.de/~benner
Towards Fast, Accurate and Reproducible LU Factorization
Towards Fast, Accurate and Reproducible LU Factorization Roman Iakymchuk 1, David Defour 2, and Stef Graillat 3 1 KTH Royal Institute of Technology, CSC, CST/PDC 2 Université de Perpignan, DALI LIRMM 3
MAGMA MIC 1.0: Linear Algebra Library for Intel Xeon Phi Coprocessors
MAGMA MIC 1.0: Linear Algebra Library for Intel Xeon Phi Coprocessors J. Dongarra, M. Gates, A. Haidar, Y. Jia, K. Kabir, P. Luszczek, and S. Tomov University of Tennessee, Knoxville 05 / 03 / 2013 MAGMA:
The Design and Implementation of the Parallel. Out-of-core ScaLAPACK LU, QR and Cholesky. Factorization Routines
The Design and Implementation of the Parallel Out-of-core ScaLAPACK LU, QR and Cholesky Factorization Routines Eduardo D Azevedo Jack Dongarra Abstract This paper describes the design and implementation
Lecture 13 Stability of LU Factorization; Cholesky Factorization. Songting Luo. Department of Mathematics Iowa State University
Lecture 13 Stability of LU Factorization; Cholesky Factorization Songting Luo Department of Mathematics Iowa State University MATH 562 Numerical Analysis II ongting Luo ( Department of Mathematics Iowa
Overview of Dense Numerical Linear Algebra Libraries
Overview of Dense Numerical Linear Algebra Libraries BLAS: kernel for dense linear algebra LAPACK: sequen;al dense linear algebra ScaLAPACK: parallel distributed dense linear algebra And Beyond Linear
Review Questions REVIEW QUESTIONS 71
REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of
Parallel Transposition of Sparse Data Structures
Parallel Transposition of Sparse Data Structures Hao Wang, Weifeng Liu, Kaixi Hou, Wu-chun Feng Department of Computer Science, Virginia Tech Niels Bohr Institute, University of Copenhagen Scientific Computing
Matrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix
Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Matrix Operations Matrix Addition and Matrix Scalar Multiply Matrix Multiply Matrix
Jacobi-Based Eigenvalue Solver on GPU. Lung-Sheng Chien, NVIDIA
Jacobi-Based Eigenvalue Solver on GPU Lung-Sheng Chien, NVIDIA lchien@nvidia.com Outline Symmetric eigenvalue solver Experiment Applications Conclusions Symmetric eigenvalue solver The standard form is
Elementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
Porting a sphere optimization program from LAPACK to ScaLAPACK
Porting a sphere optimization program from LAPACK to ScaLAPACK Mathematical Sciences Institute, Australian National University. For presentation at Computational Techniques and Applications Conference
MAGMA. Matrix Algebra on GPU and Multicore Architectures. Mark Gates. February 2012
MAGMA Matrix Algebra on GPU and Multicore Architectures Mark Gates February 2012 1 Hardware trends Scale # cores instead of clock speed Hardware issue became software issue Multicore Hybrid 1.E+07 1e7
Abstract. the diæculties inherent in producing correct codes for networks of heterogeneous processors. We
LAPACK Working Note 95 ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance æ J. Choi y, J. Demmel z, I. Dhillon z, J. Dongarra x, S. Ostrouchov
Math 3191 Applied Linear Algebra
Math 191 Applied Linear Algebra Lecture 8: Inverse of a Matrix Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/0 Announcements We will not make it to section. tonight,
Computing least squares condition numbers on hybrid multicore/gpu systems
Computing least squares condition numbers on hybrid multicore/gpu systems M. Baboulin and J. Dongarra and R. Lacroix Abstract This paper presents an efficient computation for least squares conditioning
NAG Library Routine Document F07HAF (DPBSV)
NAG Library Routine Document (DPBSV) Note: before using this routine, please read the Users Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent
Matrix Computations: Direct Methods II. May 5, 2014 Lecture 11
Matrix Computations: Direct Methods II May 5, 2014 ecture Summary You have seen an example of how a typical matrix operation (an important one) can be reduced to using lower level BS routines that would
Level-3 BLAS on a GPU
Level-3 BLAS on a GPU Picking the Low Hanging Fruit Francisco Igual 1 Gregorio Quintana-Ortí 1 Robert A. van de Geijn 2 1 Departamento de Ingeniería y Ciencia de los Computadores. University Jaume I. Castellón
TR A Comparison of the Performance of SaP::GPU and Intel s Math Kernel Library (MKL) for Solving Dense Banded Linear Systems
TR-0-07 A Comparison of the Performance of ::GPU and Intel s Math Kernel Library (MKL) for Solving Dense Banded Linear Systems Ang Li, Omkar Deshmukh, Radu Serban, Dan Negrut May, 0 Abstract ::GPU is a
On aggressive early deflation in parallel variants of the QR algorithm
On aggressive early deflation in parallel variants of the QR algorithm Bo Kågström 1, Daniel Kressner 2, and Meiyue Shao 1 1 Department of Computing Science and HPC2N Umeå University, S-901 87 Umeå, Sweden
CS 594 Scientific Computing for Engineers Spring Dense Linear Algebra. Mark Gates
CS 594 Scientific Computing for Engineers Spring 2017 Dense Linear Algebra Mark Gates mgates3@icl.utk.edu http://www.icl.utk.edu/~mgates3/ 1 Outline Legacy Software BLAS LINPACK LAPACK ScaLAPACK New Software
Matrix Algebra. Matrix Algebra. Chapter 8 - S&B
Chapter 8 - S&B Algebraic operations Matrix: The size of a matrix is indicated by the number of its rows and the number of its columns. A matrix with k rows and n columns is called a k n matrix. The number
Strassen s Algorithm for Tensor Contraction
Strassen s Algorithm for Tensor Contraction Jianyu Huang, Devin A. Matthews, Robert A. van de Geijn The University of Texas at Austin September 14-15, 2017 Tensor Computation Workshop Flatiron Institute,
Efficient algorithms for symmetric tensor contractions
Efficient algorithms for symmetric tensor contractions Edgar Solomonik 1 Department of EECS, UC Berkeley Oct 22, 2013 1 / 42 Edgar Solomonik Symmetric tensor contractions 1/ 42 Motivation The goal is to
A Massively Parallel Eigenvalue Solver for Small Matrices on Multicore and Manycore Architectures
A Massively Parallel Eigenvalue Solver for Small Matrices on Multicore and Manycore Architectures Manfred Liebmann Technische Universität München Chair of Optimal Control Center for Mathematical Sciences,
Performance optimization of WEST and Qbox on Intel Knights Landing
Performance optimization of WEST and Qbox on Intel Knights Landing Huihuo Zheng 1, Christopher Knight 1, Giulia Galli 1,2, Marco Govoni 1,2, and Francois Gygi 3 1 Argonne National Laboratory 2 University
Multicore Parallelization of Determinant Quantum Monte Carlo Simulations
Multicore Parallelization of Determinant Quantum Monte Carlo Simulations Andrés Tomás, Che-Rung Lee, Zhaojun Bai, Richard Scalettar UC Davis SIAM Conference on Computation Science & Engineering Reno, March
Outline. Recursive QR factorization. Hybrid Recursive QR factorization. The linear least squares problem. SMP algorithms and implementations
Outline Recursie QR factorization Hybrid Recursie QR factorization he linear least squares problem SMP algorithms and implementations Conclusions Recursion Automatic ariable blocking (e.g., QR factorization)
6 Linear Systems of Equations
6 Linear Systems of Equations Read sections 2.1 2.3, 2.4.1 2.4.5, 2.4.7, 2.7 Review questions 2.1 2.37, 2.43 2.67 6.1 Introduction When numerically solving two-point boundary value problems, the differential
Review : Powers of a matrix
Review : Powers of a matrix Given a square matrix A and a positive integer k, we define A k = AA A } {{ } k times Note that the multiplications AA, AAA,... make sense. Example. Suppose A=. Then A 0 2 =
Linear Algebra (Review) Volker Tresp 2018
Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c
Fall Inverse of a matrix. Institute: UC San Diego. Authors: Alexander Knop
Fall 2017 Inverse of a matrix Authors: Alexander Knop Institute: UC San Diego Row-Column Rule If the product AB is defined, then the entry in row i and column j of AB is the sum of the products of corresponding
Cluster Computing: Updraft. Charles Reid Scientific Computing Summer Workshop June 29, 2010
Cluster Computing: Updraft Charles Reid Scientific Computing Summer Workshop June 29, 2010 Updraft Cluster: Hardware 256 Dual Quad-Core Nodes 2048 Cores 2.8 GHz Intel Xeon Processors 16 GB memory per
Tips Geared Towards R. Adam J. Suarez. Arpil 10, 2015
Tips Geared Towards R Departments of Statistics North Carolina State University Arpil 10, 2015 1 / 30 Advantages of R As an interpretive and interactive language, developing an algorithm in R can be done
Math 343 Midterm I Fall 2006 sections 002 and 003 Instructor: Scott Glasgow
Math 343 Midterm I Fall 006 sections 00 003 Instructor: Scott Glasgow 1 Assuming A B are invertible matrices of the same size, prove that ( ) 1 1 AB B A 1 = (11) B A 1 1 is the inverse of AB if only if
GPU accelerated Arnoldi solver for small batched matrix
15. 09. 22 GPU accelerated Arnoldi solver for small batched matrix Samsung Advanced Institute of Technology Hyung-Jin Kim Contents - Eigen value problems - Solution - Arnoldi Algorithm - Target - CUDA
Chapter 2 Notes, Linear Algebra 5e Lay
Contents.1 Operations with Matrices..................................1.1 Addition and Subtraction.............................1. Multiplication by a scalar............................ 3.1.3 Multiplication
Lightweight Superscalar Task Execution in Distributed Memory
Lightweight Superscalar Task Execution in Distributed Memory Asim YarKhan 1 and Jack Dongarra 1,2,3 1 Innovative Computing Lab, University of Tennessee, Knoxville, TN 2 Oak Ridge National Lab, Oak Ridge,
NAG Library Routine Document F08VAF (DGGSVD)
NAG Library Routine Document (DGGSVD) Note: before using this routine, please read the Users Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent
NAG Library Routine Document F08PNF (ZGEES)
F08 Least-squares and Eigenvalue Problems (LAPACK) F08PNF NAG Library Routine Document F08PNF (ZGEES) Note: before using this routine, please read the Users Note for your implementation to check the interpretation
DM559 Linear and Integer Programming. Lecture 3 Matrix Operations. Marco Chiarandini
DM559 Linear and Integer Programming Lecture 3 Matrix Operations Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline and 1 2 3 and 4 2 Outline and 1 2
The LINPACK Benchmark in Co-Array Fortran J. K. Reid Atlas Centre, Rutherford Appleton Laboratory, Chilton, Didcot, Oxon OX11 0QX, UK J. M. Rasmussen
The LINPACK Benchmark in Co-Array Fortran J. K. Reid Atlas Centre, Rutherford Appleton Laboratory, Chilton, Didcot, Oxon OX11 0QX, UK J. M. Rasmussen and P. C. Hansen Department of Mathematical Modelling,
Algorithms and Methods for Fast Model Predictive Control
Algorithms and Methods for Fast Model Predictive Control Technical University of Denmark Department of Applied Mathematics and Computer Science 13 April 2016 Background: Model Predictive Control Model
MATH 323 Linear Algebra Lecture 6: Matrix algebra (continued). Determinants.
MATH 323 Linear Algebra Lecture 6: Matrix algebra (continued). Determinants. Elementary matrices Theorem 1 Any elementary row operation σ on matrices with n rows can be simulated as left multiplication
NAG Fortran Library Routine Document F04CFF.1
F04 Simultaneous Linear Equations NAG Fortran Library Routine Document Note: before using this routine, please read the Users Note for your implementation to check the interpretation of bold italicised
3.4 Elementary Matrices and Matrix Inverse
Math 220: Summer 2015 3.4 Elementary Matrices and Matrix Inverse A n n elementary matrix is a matrix which is obtained from the n n identity matrix I n n by a single elementary row operation. Elementary
Direct Self-Consistent Field Computations on GPU Clusters
Direct Self-Consistent Field Computations on GPU Clusters Guochun Shi, Volodymyr Kindratenko National Center for Supercomputing Applications University of Illinois at UrbanaChampaign Ivan Ufimtsev, Todd
PRECONDITIONING IN THE PARALLEL BLOCK-JACOBI SVD ALGORITHM
Proceedings of ALGORITMY 25 pp. 22 211 PRECONDITIONING IN THE PARALLEL BLOCK-JACOBI SVD ALGORITHM GABRIEL OKŠA AND MARIÁN VAJTERŠIC Abstract. One way, how to speed up the computation of the singular value
Matrices and systems of linear equations
Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T.
LAPACK Introduction. Zhaojun Bai University of California/Davis. James Demmel University of California/Berkeley
93 LAPACK Zhaojun Bai University of California/Davis James Demmel University of California/Berkeley Jack Dongarra University of Tennessee Julien Langou University of Colorado Denver Jenny Wang University
Scientific Computing: Dense Linear Systems
Scientific Computing: Dense Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 February 9th, 2012 A. Donev (Courant Institute)
Tile QR Factorization with Parallel Panel Processing for Multicore Architectures
Tile QR Factorization with Parallel Panel Processing for Multicore Architectures Bilel Hadri, Hatem Ltaief, Emmanuel Agullo, Jack Dongarra Department of Electrical Engineering and Computer Science, University
Review of Basic Concepts in Linear Algebra
Review of Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University September 7, 2017 Math 565 Linear Algebra Review September 7, 2017 1 / 40 Numerical Linear Algebra
MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian.
MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian. Spanning set Let S be a subset of a vector space V. Definition. The span of the set S is the smallest subspace W V that contains S. If
The Algorithm of Multiple Relatively Robust Representations for Multi-Core Processors
Aachen Institute for Advanced Study in Computational Engineering Science Preprint: AICES-2010/09-4 23/September/2010 The Algorithm of Multiple Relatively Robust Representations for Multi-Core Processors
Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn
Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear
The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and
Home Search Collections Journals About Contact us My IOPscience The ELPA library: scalable parallel eigenvalue solutions for electronic structure theory and computational science This content has been
CS 594 Scientific Computing for Engineers Spring Dense Linear Algebra. Mark Gates
CS 594 Scientific Computing for Engineers Spring 2018 Dense Linear Algebra Mark Gates mgates3@icl.utk.edu http://www.icl.utk.edu/~mgates3/ 1 Outline Legacy Software BLAS LINPACK LAPACK ScaLAPACK New Software
Parallelization Strategies for Density Matrix Renormalization Group algorithms on Shared-Memory Systems
Parallelization Strategies for Density Matrix Renormalization Group algorithms on Shared-Memory Systems G. Hager HPC Services, Computing Center Erlangen, Germany E. Jeckelmann Theoretical Physics, Univ.
A NOVEL PARALLEL QR ALGORITHM FOR HYBRID DISTRIBUTED MEMORY HPC SYSTEMS
A NOVEL PARALLEL QR ALGORITHM FOR HYBRID DISTRIBUTED MEMORY HPC SYSTEMS ROBERT GRANAT, BO KÅGSTRÖM, AND DANIEL KRESSNER Abstract A novel variant of the parallel QR algorithm for solving dense nonsymmetric
Math 4377/6308 Advanced Linear Algebra
2.3 Composition Math 4377/6308 Advanced Linear Algebra 2.3 Composition of Linear Transformations Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/ jiwenhe/math4377
ICS 6N Computational Linear Algebra Matrix Algebra
ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24 Matrix Consider an m n matrix
Parallel Genetic Algorithms
Parallel Genetic Algorithms Timothy H. Kaiser, Ph.D. Introduction Project Purpose What is a Genetic Algorithm? Serial Algorithm Modes of Parallelization My All to All Example Problems Results Future Direction
ON THE FUTURE OF HIGH PERFORMANCE COMPUTING: HOW TO THINK FOR PETA AND EXASCALE COMPUTING
ON THE FUTURE OF HIGH PERFORMANCE COMPUTING: HOW TO THINK FOR PETA AND EXASCALE COMPUTING JACK DONGARRA UNIVERSITY OF TENNESSEE OAK RIDGE NATIONAL LAB What Is LINPACK? LINPACK is a package of mathematical
Linear Systems and Matrices
Department of Mathematics The Chinese University of Hong Kong 1 System of m linear equations in n unknowns (linear system) a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.......
SOLUTION of linear systems of equations of the form:
Proceedings of the Federated Conference on Computer Science and Information Systems pp. Mixed precision iterative refinement techniques for the WZ factorization Beata Bylina Jarosław Bylina Institute of
Solution to Laplace Equation using Preconditioned Conjugate Gradient Method with Compressed Row Storage using MPI
Solution to Laplace Equation using Preconditioned Conjugate Gradient Method with Compressed Row Storage using MPI Sagar Bhatt Person Number: 50170651 Department of Mechanical and Aerospace Engineering,
Linear Independence x
Linear Independence A consistent system of linear equations with matrix equation Ax = b, where A is an m n matrix, has a solution set whose graph in R n is a linear object, that is, has one of only n +