6.3 Basic Linear Algebra Subprograms (BLAS1)

Transcription

1 6.3 Basi Linear Algebra Subprograms (BLAS1) A. Purpose This is a set of subroutine and funtion subprograms for basi mathematial operations on a single vetor or a pair of vetors. The operations provided are those ommonly used in algorithms for numerial linear algebra problems, e.g., problems involving systems of equations, least-squares, matrix eigenvalues, optimization, et. B. Usage Desribed below under B.1 through B.13 are: B.1 Vetor Arguments B.2 Dot Produt SDOT, DDOT, CDOTC, CDOTU, DSDOT, SDSDOT B.3 Salar times a Vetor Plus a Vetor SAXPY, DAXPY, CAXPY B.4 Set up Givens Rotation SROTG, DROTG B.5 Apply Givens Rotation SROT, DROT B.6 Set up Modified Rotation SROTMG, DROTMG B.7 Apply Modified Rotation SROTM, DROTM.. 3 B.8 Copy X into Y SCOPY, DCOPY, CCOPY... 4 B.9 Swap X and Y SSWAP, DSWAP, CSWAP B.10 Eulidean Norm SNRM2, DNRM2, SCNRM B.11 Sum of Absolute Values SASUM, DASUM, SCASUM B.12 Constant Times a Vetor SSCAL, DSCAL, CSCAL, CSSCAL B.13 Index of Element Having Maximum Absolute Value ISAMAX, IDAMAX, ICAMAX B.1 Vetor Arguments For subprograms of this pakage a vetor is speified by three arguments, say N, SX, and INCX, where N denotes the number of elements in the vetor, SX (or DX or CX) the vetor, and identifies the array ontaining INCX is the (signed) storage inrement between suessive elements of the vetor. Let x i, i = 1,..., N, denote the vetor stored in the array SX(). Within a subprogram of this pakage the array argument, SX, will be delared as SX( ) In the ommon ase of INCX = 1, x i is stored in SX(i). More generally, if INCX 0, x i is stored in SX(1 + (i 1) INCX), and if INCX < 0, x i is stored in X(1 + (N i) INCX ) Calif. Inst. of Tehnology, 2015 Math à la Carte, In. With indexing as speified above, looping operations within the subprograms are performed in the order, 2,..., N. Only positive values of INCX are allowed for subprograms that have a single vetor argument. For subprograms having two vetor arguments, the two inrement parameters, INCX and INCY, may independently be positive, zero, or negative. Due to the Fortran 77 rules for argument assoiation and array storage, the atual argument playing the role of SX is not limited to being a singly dimensioned array. It may be an array of any number of dimensions, or it may be an array element with any number of subsripts. The standard does not permit this argument to be a simple variable, however. If the atual argument is an array element, i.e., a subsripted array name, then it identifies x 1, the first element of the vetor, if INCX 0, and x N, the last element of the vetor, if INCX < 0. The most ommon reason for using an inrement value different from one is to operate on a row of a matrix. For example, if an array is delared as A(5, 10) The third row of A(,) begins at element A(3,1) and has a storage spaing of 5 (double preision) storage loations between suessive elements. Thus this row-vetor of length 10 would be desribed by the parameter triple (N, DX, INCX) = (10, A(3,1), 5) in alls to subprograms of this pakage. See Setion C for examples illustrating the speifiations of vetor arguments. In the following subprogram desriptions we assume the following delarations have been made for any subprograms that use these variable names INTEGER N, INCX, INCY SX(mx), SY(my) COMPLEX CX(mx), CY(my) DX(mx), DY(my) In the ommon ase of INCX = 1, mx must satisfy mx N. More generally, mx must satisfy mx 1 + (N 1) INCX. Similarly, my must be onsistent with N and INCY. In Setions B.2 through B.13 we use the onvention that x denotes the vetor ontained in the storage array SX(), July 11, 2015 Basi Linear Algebra Subprograms (BLAS1) 6.3 1

2 DX(), or CX(), a denotes the salar value ontained in SA, DA, or CA, et. B.2 Dot Produt Subprograms SDOT, SDSDOT, SB, SW COMPLEX DDOT, DSDOT, DW CDOTC, CDOTU, CW The first four subprograms eah ompute w = x i y i Single preision: SW = SDOT(N, SX, INCX, SY, INCY) Double preision: DW = DDOT(N, DX, INCX, DY, INCY) Single preision data. Uses double preision arithmeti internally and returns a double preision result: DW = DSDOT(N, SX, INCX, SY, INCY) Complex (unonjugated): CW = CDOTU(N, CX, INCX, CY, INCY) CDOTC omputes w = x i y i where x i denotes the omplex onjugate of the given x i. This is the usual inner produt of omplex N-spae. CW = CDOTC(N, CX, INCX, CY, INCY) SDSDOT omputes w = b + x i y i using single preision data, double preision internal arithmeti, and onverting the final result to single preision: SW = SDSDOT(N, SB, SX, INCX, SY, INCY) In eah of the above six subprograms the value of the summation from 1 to N will be set to zero if N 0. B.3 Salar Times a Vetor Plus a Vetor SA COMPLEX CA DA Given a salar, a, and vetors, x and y, eah of these subroutines replaes y by ax + y. If a = 0 or N 0, eah subroutine returns, doing no omputation. Single preision: CALL SAXPY (N, SA, SX, INCX, SY, INCY) Double preision: CALL DAXPY (N, DA, DX, INCX, DY, INCY) Complex: CALL CAXPY (N, CA, CX, INCX, CY, INCY) B.4 Construt a Givens Plane Rotation SA, SB, SC, SS Single preision: Double preision: DA, DB, DC, DS CALL SROTG (SA, SB, SC, SS) CALL DROTG (DA, DB, DC, DS) Given a and b, eah of these subroutines omputes and s, satisfying s a r = s b 0 subjet to 2 + s 2 = 1 and r 2 = a 2 + b 2. Thus the matrix involving and s is an orthogonal (rotation) matrix that transforms the seond omponent of the vetor a, b t to zero. This matrix is used in ertain least-squares and eigenvalue algorithms. If r = 0 the subroutine sets = 1 and s = 0. Otherwise the sign of r is set so that sgn(r) = sgn(a) if a > b, and sgn(r) = sgn(b) if a b. Then, = a/r and s = b/r. Besides setting and s, the subroutine stores r in plae of a and another number, z, in plae of b. The number z is rarely needed. See 1 4 in Setion D for a desription of z. These subroutines are designed to avoid extraneous overflow or underflow in ases where r 2 is outside the exponent range of the omputer arithmeti but r is within the range Basi Linear Algebra Subprograms (BLAS1) July 11, 2015

3 B.5 Apply a Plane Rotation SC, SS Single preision: Double preision: DC, DS CALL SROT (N, SX, INCX, SY, INCY, SC, SS) CALL DROT (N, DX, INCX, DY, INCY, DC, DS) Given vetors, x and y, and salars, and s, this subroutine replaes the 2 N matrix x t y t by the 2 N matrix s s x t y t. If N 0 or if = 1 and s = 0, these subroutines return, doing no omputation. B.6 Construt a Modified Givens Transformation SD1, SD2, SX1, SX2, SPARAM(5) DPARAM(5) Single preision: Double preision: DD1, DD2, DX1, DX2, CALL SROTMG (SD1, SD2, SX1, SX2, SPARAM) CALL DROTMG (DD1, DD2, DX1, DX2, DPARAM) The input quantities d 1, d 2, x 1, and x 2, define a 2-vetor w 1 w 2 t in partitioned form as = w 2 w1 d 1/ d 1/2 2 x1. x 2 The subroutine determines the modified Givens rotation matrix H that transforms x 2 and thus w 2 to zero. It also replaes d 1, d 2, and x 1 with δ 1, δ 2, and ξ 1, respetively. These quantities satisfy ω 0 = = δ 1/ δ 1/2 2 δ 1/ δ 1/2 2 x1 H x 2 ξ1 0 with ω = ±(w w 2 2) 1/2. A representation of the matrix H will be stored by this subroutine into SPARAM() or DPARAM() for subsequent use by subroutines SROTM or DROTM. See the Appendix in 1 for more details on the omputation and storage of H. Most of the time the matrix H will be onstruted to have two elements equal to +1 or 1. Thus multipliation of a 2-vetor by H an be programmed to be faster than multipliation by a general 2 2 matrix. This is the motivation for using a modified Givens matrix rather than a standard Givens matrix; however, these matrix multipliations must represent a very signifiant perentage of the exeution time of an appliation program in order for the extra omplexity of using the modified Givens matrix to be worthwhile. B.7 Apply a Modified Givens Transformation SPARAM(5) Single preision: Double preision:, DPARAM(5) CALL SROTM (N, SX, INCX, SY, INCY, SPARAM) CALL DROTM (N, DX, INCX, DY, INCY, DPARAM) Given vetors, x and y, and a representation of an H matrix onstruted by SROTMG or DROTMG in SPARAM() or DPARAM(), this subroutine replaes the 2 N matrix x t y t x t by H y t. Due to the speial form of the matrix, H, this matrix multipliation generally requires only 2N multipliations and 2N additions rather than the 4N multipliations and 2N additions that would be required if H were an arbitrary 2 2 matrix. If N 0 or H is the identity matrix, these subroutines return immediately. July 11, 2015 Basi Linear Algebra Subprograms (BLAS1) 6.3 3

4 B.8 Copy a Vetor x to y Single preision: CALL SCOPY (N, SX, INCX, SY, INCY) Double preision: CALL DCOPY (N, DX, INCX, DY, INCY) Complex: CALL CCOPY (N, CX, INCX, CY, INCY) Eah of these subroutines opies the vetor x to y. If N 0 the subroutine returns immediately. B.9 Swap Vetors x and y Single preision: CALL SSWAP (N, SX, INCX, SY, INCY) Double preision: CALL DSWAP (N, DX, INCX, DY, INCY) Complex: CALL CSWAP (N, CX, INCX, CY, INCY) This subroutine interhanges the vetors x and y. If N 0, the subroutine returns immediately. B.10 Eulidean Norm of a Vetor SNRM2, SCNRM2, SW Single preision: Double preision: DNRM2, DW SW = SNRM2(N, SX, INCX) DW = DNRM2(N, DX, INCX) Complex data, result: SW = SCNRM2(N, CX, INCX) Eah of these subprograms omputes N 1/2 w = x i 2. If N 0, the result is set to zero. These subprograms are designed to avoid overflow or underflow in ases in whih w 2 is outside the exponent range of the omputer arithmeti but w is within the range. B.11 Sum of Magnitude of Vetor Components SASUM, SCASUM, SW SASUM and DASUM ompute w = x i. Single preision: Double preision: DASUM, DW SW = SASUM(N, SX, INCX) DW = DASUM(N, DX, INCX) Given a omplex vetor, x, SCASUM omputes a result for the expression: w = Rx i + Ix i. SW = SCASUM(N, CX, INCX) If N 0 these subprograms set the result to zero. B.12 Vetor Saling SA COMPLEX CA DA Given a salar, a, and vetor, x, eah of these subroutines replaes the vetor, x, by the produt ax. Single preision: CALL SSCAL (N, SA, SX, INCX) Double preision: Complex: CALL DSCAL (N, DA, DX, INCX) CALL CSCAL (N, CA, CX, INCX) Given a and omplex x, the subroutine CSSCAL replaes x by the produt ax. CALL CSSCAL (N, SA, CX, INCX) If N 0 these subroutines return immediately Basi Linear Algebra Subprograms (BLAS1) July 11, 2015

5 B.13 Find Vetor Component of Largest Magnitude INTEGER ISAMAX, IDAMAX, ICAMAX, IMAX ISAMAX and IDAMAX eah determine the smallest i suh that x i = max{ x j : j = 1,..., N} vetor, integer result: IMAX = ISAMAX(N, SX, INCX) Double preision vetor, integer result: IMAX = IDAMAX(N, DX, INCX) Given a omplex vetor, x, ICAMAX determines the smallest i suh that Rx i + Ix i = max{ Rx j + Ix j : j = 1,..., N} IMAX = ICAMAX(N, CX, INCX) If N 0, eah of these subprograms sets the integer result to zero. C. Examples and Remarks The program, DRDBLAS1, and its output, ODDBLAS1, illustrate the use of various BLAS1 subprograms to ompute matrix-vetor and matrix-matrix produts. Problems (1) and (2) show two ways of omputing the matrix-vetor produt, Ab. Using DDOT aesses the elements of A by rows. Using DAXPY aesses A by olumns. The olumn ordering has an effiieny advantage on virtual memory systems sine Fortran stores arrays by olumns and there will therefore be fewer page faults. Problem (3) illustrates multipliation by the transpose of a matrix without transposing the matrix in storage. Problem (4) illustrates matrix-matrix multipliation. This operation ould also be programmed to use DAXPY rather than DDOT. These examples also illustrate the feature that matrix dimensions and storage array dimensions need not be the same. Thus, in DRDBLAS1, a 2 3 matrix A is stored in a 5 10 storage array A(,). The program, DRDBLAS2, with output, ODDBLAS2, illustrates a omplete algorithm for solving a linear leastsquares problem. The algorithm first performs sequential aumulation of data into a triangular matrix, using DROTG and DROT to build and apply Givens orthogonal transformations. It then solves the triangular system using DCOPY and DAXPY. This is a very reliable algorithm. The problem solved is the determination of oeffiients 1, 2, and 3 in the expression x + 3 exp( x) to produe a least-squares fit to the data given in XTAB() and YTAB(). Note that by hanging dimension parameters, and the statements assigning values to the array W(), DRDBLAS2 ould be altered to solve any speifi linear least-squares problem. D. Funtional Desription This set of subprograms is desribed in more detail in 1. For disussion of the standard and modified Givens orthogonal transformations and their use in least-squares omputations see 5. There is an error in 1 in the disussion of the parameter, z, whih is returned by SROTG or DROTG. This error was orreted by 3, after whih 4 orreted an error in 3. Referenes 1. C. L. Lawson, R. J. Hanson, D. R. Kinaid, and F. T. Krogh, Basi Linear Algebra Subprograms for Fortran usage, ACM Trans. on Math. Software 5, 3 (Sept. 1979) C. L. Lawson, R. J. Hanson, D. R. Kinaid, and F. T. Krogh, Algorithm 539: Basi Linear Algebra Subprograms for Fortran usage F1, ACM Trans. on Math. Software 5, 3 (Sept. 1979) David S. Dodson and Roger G. Grimes, Remark on Algorithm 539: Basi Linear Algebra Subprograms for Fortran usage F1, ACM Trans. on Math. Software 8, 4 (De. 1982) David S. Dodson, Corrigendum: Remark on Algorithm 539: Basi Linear Algebra Subroutines for FOR- TRAN usage, ACM Trans. on Math. Software 9, 1 (Marh 1983) Charles L. Lawson and Rihard J. Hanson, Solving Least-Squares Problems, Prentie-Hall, Englewood Cliffs, N. J. (1974) 340 pages. 6. Fred T. Krogh, On the Use of Assembly Code for Heavily Used Modules in Linear Algebra. Internal Tehnial Memorandum 303, Jet Propulsion Laboratory, Pasadena, CA (May 1972). 7. R. Hanson, F. Krogh, and C. Lawson, A proposal for standard linear algebra subprograms. Internal Tehnial Memorandum , Jet Propulsion Laboratory (Nov. 1973). 8. Jak J. Dongarra, Jeremy Du Croz, Sven Hammarling, and Rihard J. Hanson, An extended set of FORTRAN Basi Linear Algebra Subprograms, ACM Trans. on Math. Software 14, 1 (Marh 1988) July 11, 2015 Basi Linear Algebra Subprograms (BLAS1) 6.3 5

6 9. Jak J. Dongarra, Jeremy Du Croz, Sven Hammarling, and Iain Duff, A set of level 3 Basi Linear Algebra Subprograms, ACM Trans. on Math. Software 16, 1 (Marh 1990) R. J. Hanson and F. T. Krogh, Algorithm 653: Translation of Algorithm 539: PC-BLAS Basi Linear Algebra Subprograms for FORTRAN usage with the INTEL 8087, numeri data proessor, ACM Trans. on Math. Software 13, 3 (Sept. 1987) E. Error Proedures and Restritions These subprograms do not issue any error messages. If INCX 0 in any of the subprograms having a single vetor argument the results are unpreditable. A value of N = 0 is a valid speial ase. Values of N < 0 are treated like N = 0. F. Supporting Information The soure language is ANSI Fortran 77. Eah program unit has a single entry with the same name as the program unit. No program units ontain external referenes. As a result of experiments done in 6, a BLAS-type pakage was originally proposed in 7. The pakage was subsequently developed and tested over the period as an ACM SIGNUM ommittee projet, with the final produt being announed in 1. This pakage was subsequently used as a omponent in many other numerial software pakages and optimized mahine-language versions were produed for a number of different omputer systems. This pakage was identified as BLAS when it appeared in , however it is now identified as BLAS1, or Level 1 BLAS sine publiation of BLAS2 8 for matrix-vetor operations and BLAS3 9 for matrixmatrix operations. These latter two pakages support more effiient use of vetor registers, proessor ahe, and parallel proessors than is possible in BLAS1. Adapted to Fortran 77, by C. Lawson and S. Chiu, JPL, January Replaed L2 norm routines with new versions whih avoid undeflow in all ases and whih don t use assigned go to s, F. Krogh, May All entries need one file of the same name, exept for DNRM2, SCNRM2, and SNRM2 all of whih also require AMACH. Entries CAXPY CCOPY CDOTC CDOTU CSCAL CSSCAL CSWAP DASUM DAXPY DCOPY DDOT DNRM2 DROT DROTG DROTM DROTMG DSCAL DSDOT DSWAP ICAMAX IDAMAX ISAMAX SASUM SAXPY SCASUM SCNRM2 SCOPY SDOT SDSDOT SNRM2 SROT SROTG SROTM SROTMG SSCAL SSWAP DRDBLAS1 program DRDBLAS1 >> DRDBLAS1 Krogh Minor format hange f o r C onversion. >> DRDBLAS1 Krogh Added e x t e r n a l statement. >> DRDBLAS1 Krogh Changes t o use M77CON >> DRDBLAS1 CLL Removed r e f e r e n e to DBLE( ) f u n t i o n. >> DRDBLAS1 CLL >> DRDBLAS1 CLL >> DRBLAS1 Lawson I n i t i a l Code. Demonstrate usage o f DAXPY, DCOPY, and DDOT from t h e BLAS by omputing (1) p = A b using DDOT (2) q = A b using DCOPY & DAXPY (3) r = (A Transposed ) p using DDOT (4) S = A E using DDOT D r e p l a e s? : DR?BLAS1,?AXPY,?COPY,?DOT external DDOT double preision DDOT integer M2, M3, M4, N2, N3, N4 parameter ( M2=5, M3=10, M4=12 ) Basi Linear Algebra Subprograms (BLAS1) July 11, 2015

7 parameter ( N2=2, N3=3, N4=4 ) integer I, J double preision A(M2,M3), E(M3,M4), S (M2,M4) double preision B(M3), P(M2), Q(M2), R(M3) double preision ZERO( 1 ) data ZERO( 1 ) / 0. 0 d0 / data (A( 1, J ), J=1,N3) / 2. 0 d0, 4.0d0, 3. 0 d0 / data (A( 2, J ), J=1,N3) / 5.0d0, 2.0d0, 6. 0 d0 / data (B( J ), J=1,N3) / 7. 0 d0, 3.0d0, 5. 0 d0 / data (E( 1, J ), J=1,N4) / 4.0d0, 2. 0 d0, 3. 0 d0, 6.0d0 / data (E( 2, J ), J=1,N4) / 7. 0 d0, 5. 0 d0, 6.0d0, 3.0d0 / data (E( 3, J ), J=1,N4) / 3. 0 d0, 4. 0 d0, 2.0d0, 5. 0 d0 / 1. p = A b using DDOT do 10 I = 1, N2 P( I ) = DDOT(N3,A( I, 1 ),M2, B, 1 ) 10 ontinue 2. q = A b using DCOPY and DAXPY a l l DCOPY(N2,ZERO, 0,Q, 1 ) do 20 J = 1, N3 a l l DAXPY(N2,B( J ),A( 1, J ), 1,Q, 1 ) 20 ontinue 3. r = (A Transposed ) p using DDOT do 30 J = 1, N3 R( J ) = DDOT(N2,A( 1, J ), 1,P, 1 ) 30 ontinue 4. S = A E using DDOT do 50 I = 1, N2 do 40 J = 1, N4 S ( I, J ) = DDOT(N3,A( I, 1 ),M2,E( 1, J ), 1 ) 40 ontinue 50 ontinue print, DRDBLAS1.. Demo d r i v e r f o r DAXPY, DCOPY, and DDOT print (/ P( ) =, 7x, 4 f 8. 1 ), (P( J ), J=1,N2) print (/ Q( ) =,7 x, 4 f 8. 1 ), (Q( J ), J=1,N2) print (/ R( ) =,7 x, 4 f 8. 1 ), (R( J ), J=1,N3) print (/ S (, ) = ) do 60 I = 1,N2 print ( Row, i2, 5 x, 4 f 8. 1 ), I, ( S ( I, J ), J=1,N4) 60 ontinue stop end July 11, 2015 Basi Linear Algebra Subprograms (BLAS1) 6.3 7

8 ODDBLAS1 DRDBLAS1.. Demo d r i v e r f o r DAXPY, DCOPY, and DDOT P( ) = Q( ) = R( ) = S (, ) = Row Row DRDBLAS2 program DRDBLAS2 >> DRDBLAS2 Krogh Minor format hange f o r C onversion. >> DRDBLAS2 Krogh Changes t o use M77CON >> DRDBLAS2 CLL >> Lawson I n i t i a l Code. Demonstrates t h e use o f BLAS s u b r o u t i n e s DROTG, DROT, DAXPY, and DCOPY to implement an a l g o r i t h m f o r s o l v i n g a l i n e a r l e a s t s q u a r e s problem using s e q u e n t i a l aumulation o f t h e data and Givens o r t h o g o n a l t r a n s f o r m a t i o n s. YTAB( ) o n t a i n s rounded v a l u e s o f X + 3 Exp( X) D r e p l a e s? : DR?BLAS2,?ROTG,?ROT,?AXPY,?COPY integer MC, MC1, MXY parameter ( MC=3, MC1=MC+1, MXY=11 ) integer IXY, J, NC, NC1, NXY double preision X, XTAB(MXY), Y, YTAB(MXY), W(MC1) double preision C, RG(MC1,MC1), S double preision COEF(MC), DIV, ESTSD, ZERO( 1 ) data XTAB / 0. 0 d0,. 1 d0,. 2 d0,. 3 d0,. 4 d0,. 5 d0,. 6 d0,. 7 d0,. 8 d0,. 9 d0, 1. 0 d0 / data YTAB / d0,. 9 1 d0,. 8 6 d0,. 8 2 d0,. 8 1 d0,. 8 2 d0,. 8 5 d0,. 8 9 d0,. 9 5 d0, d0, d0 / data NXY, NC / MXY, MC / data ZERO( 1 ) / 0. 0 d0 / NC1 = NC + 1 a l l DCOPY(MC1 MC1, ZERO, 0, RG, 1) do 20 IXY = 1, NXY X = XTAB(IXY) Y = YTAB(IXY) Build new row o f A:B in W( ). W( 1 ) = 1. 0 d0 W( 2 ) = X W( 3 ) = exp( X) W( 4 ) = Y Proess W( ) i n t o R:G Basi Linear Algebra Subprograms (BLAS1) July 11, 2015

9 do 10 J = 1, NC a l l DROTG(RG( J, J ),W( J ),C, S ) a l l DROT(NC1 J,RG( J, J+1),MC1,W( J +1),1,C, S ) 10 ontinue a l l DROTG(RG(NC1,NC1),W(NC1),C, S ) 20 ontinue Begin : S o l v e t r i a n g u l a r system. a l l DCOPY(NC,RG( 1,NC1), 1,COEF, 1 ) do 30 J = NC, 1, 1 DIV = RG( J, J ) i f (DIV. eq d0 ) then print ( ERROR:ZERO DIVISOR AT J =, I2 ), J stop end i f COEF( J ) = COEF( J ) / DIV a l l DAXPY( J 1, COEF( J ),RG( 1, J ), 1,COEF, 1 ) 30 ontinue End : S o l v e t r i a n g u l a r system. print ( S o l u t i o n : COEF( ) =,3 f 8. 3 ), (COEF( J ), J=1,NC) ESTSD = abs (RG(NC1, NC1) ) / sqrt (DBLE(NXY NC) ) print (/ Estimated Std. Dev. o f data e r r o r s =, f 9. 5 ), ESTSD stop end ODDBLAS2 S o l u t i o n : COEF( ) = Estimated Std. Dev. o f data e r r o r s = July 11, 2015 Basi Linear Algebra Subprograms (BLAS1) 6.3 9