Level-2 BLAS. Matrix-Vector operations with O(n 2 ) operations (sequentially) BLAS-Notation: S --- single precision G E general matrix M V --- vector

Transcription

1 evel-2 BS trx-vector opertos wth 2 opertos sequetlly BS-Notto: S --- sgle precso G E geerl mtrx V --- vector defes SGEV, mtrx-vector product: r y r α x β r y ther evel-2 BS: Solvg trgulr system x wth trgulr mtrx

2 evel-3 BS trx-trx opertos wth 3 opertos sequetlly BS-Notto: S --- sgle precso G E geerl mtrx --- mtrx defes SGE, mtrx-mtrx product: C α B β C PCK suroutes for solvg ler equtos, lest squres prolems, QR-decomposto, egevlues, sgulr vlues sed o BS 2

3 3 2.2 lyss of trx-vector- Product m m m IR c IR IR,,,,...,,..., 2.2. Vectorzto m m m m m m m m m m c c DT-products of legth m m SXPY s of legth GXPY

4 Pseudocode: -form: c ; for,, for,,m c c ed ed DT product c, Dot product of -th row of wth vector c 4

5 Pseudocode: -form: c ; for,,m for,, c c ed ed c c SXPY SXPY updtg vector c wth -th colum of GXPY GXPY: Sequece of SXPY s relted to the sme vector dvtge: vector c, tht s updted, c e ept fst memory! 5 No ddtol dt trsfer.

6 GXPY SXPY: x : x αy GXPY: x for ed x : x : x α y Seres of SXPYs regrdg the sme vector x. dvtge: ess dt trsfer! 6

7 J s <, > <, m > Prllelzto y uldg locs Reduce mtrx-vector product o smller mtrx-vector products. {,2,..., } I I I {,2,..., m} J J 2 J S 2 R I dsuct: J I Use 2-dmesol rry of processors P rs. P rs gets mtrx loc rs :I r,j s, s :J s, c r :ci r. J for for I r rs s J s c r I r c r S S rs s s s c s r 7

8 Pseudocode for r,,r for s,,s c r s rs s ; ed ed Smll, depedet mtrx-vector products. No commucto ecessry durg computtos! for r,,r c r ; for s,,s c r c r c r s ; ed ed Blocwse collecto d ddto of vectors. Rowwse commucto! F-. 8

9 9 Specl cse: S c 2 2 P P 2.. No commucto ecessry etwee processor P,,P R The computto of s vectorzle y GXPY s. R : c re depedet. The collecto of prtl results from processor P,,P r. F-. Fl sum oe processor: vectorzle y GXPY s.

10 Rules Ier loops of progrm should e smple, vectorzle 2 uter loop of progrmm should e susttl, depedet, prllelzle for susttl d prllelzle for. ed smple, vectorzle ed 3 Reuse of dt Cche, mml dt trsfer, locg 2

11 c for Bded trx β β β β,,,, 22, β β Bdwdth β symmetrc 2β dgols: m dg. β sudg. β superdg. β: trdgol,,,,, 2, β β β β β β

12 22,,,,, 2, β β β β β β,,,,,, N N N β β β β β β Storg etres dgolwse: 2β mtrx sted of 2. row for s s,...,,, d s d s β β row dgol s β β s d s d s s [ ] { } { } [ ] r l s,,m, mx, β β [ ] { } { } [ ] s s r l s s,,m mx,,

13 Computto of the mtrx-vector product sed o ths storge scheme o vector processors: For,,: c r, s s s l r s l, s s For s -β : β For mx{-s,} : m{-s,} c c s s ed ed s Geerl trde No SXPY or For : For s mx{-β,-} : mx{β,-} c c s Prtl Dot product s ed ed 23 Sprsty less opertos, ut lso loss of effcecy

14 Bd Prllel Prttog: for ed R <, > U I c r r s l r s I r, s ; dsuct Processor P r gets rows to dex set I r :[m r, r ] order to compute ts prt of the fl vector c. Wht prt of vector does processor P r eed order to compute ts prt of c? 24

15 Necessry for I r : s : s m r l m r m r mx { β, m } mx{ m β,} r r s r r r r m { β, } m{ β, } r r Processor P r wth dex set I r eeds from the dces [ mx{, m β },m{ β} ], r r 25

16 lyss of trx-trx Product q q m m c B C B,...,,...,,...,,...,,...,,...,,, ed ed c q For For m : : * * * * * * * * * * * * * * * * * * * * m m c

17 2.3. Vectorzto -Form: lgorthm For : For : q For : m c c ; ed ed ed Dot product of legth m c B for ll, ll etres c re fully computed, oe fter other. ccess to d C s rowwse, to B columwse. 27

18 28 ther vew o the mtrx-mtrx product: m T m m T e e e e 2 trx cosdered s comto of colums or rows T T B B e e B e e B, s sum of full mtrces B y outer product of the -th colum of d the -th row of B Full x q - mtrces

19 -Form, lgorthm 2 For : q For : m For : c c ; ed ed ed c c Vector updte c. SXPY GXPY c Sequece of SXPY s for the sme vector c. C computed columwse; ccess to columwse. ccess to B columwse, ut delyed. 29

20 -Form, lgorthm 3 For : m For : q For : c c ; ed ed ed c c SXPY Vector updte c. No GXPY Sequece of SXPY s for dfferet vectors c. ccess to columwse. ccess to B delyed. C computed wth termedte vlues c whch re computed columwse. 3

21 vervew over dfferet Forms ccess to y ccess to B y Comput -to of C row row colum colum colum row row colum row row row colum colum colum Comput drect delyed delyed drect delyed delyed -to of c Vector operto Vector leght DT GXPY SXPY DT GXPY SXPY m q q m m m Better: GXPY, loger vector legth. ccess to mtrces ccordg to storge scheme rowwse or columwse 3

22 2.3.2 trx-trx Prllel R <, > U I, <, m > U K, <, q > U J r r S s s T t t. Dstrute the locs reltve to dex sets I r, K s, d J t to processor rry P rst : K s J t J t K s. I r rs c s rt I r B st Processor P rst computes smll mtrx-mtrx product. ll Processors prllel. 2 Compute sum y f- s: c rt c rt c s rt S s rs s B 32 st

23 Specl Cse S J t J t. I r r c rt I r B t I ths cse ech processor P rt c compute ts prt of c, c rt, depedetly wthout y commucto. Ech processor eeds full loc of rows of, reltve to dex set I r, d full loc of colums of B, reltve to dex set J t, order to compute c rt reltve to rows I d colums J t. 33

24 Specl Cse S J t J t. I r r c rt I r B t Especlly wth *q processors ech processor hs to compute oe DT product wth m prllel tme steps. c rt m r t F- y m q ddtol processors for ll these Dot products reduces the umer of prllel tme steps to logm. 34

25 Grulrty for BS BS: perto Formul memory Grulrty BS- XPY: 2 αxy 2 < BS-2 GEV: 2 2 αxβy BS-3 GE: 2 3 αbβc 4 2 /2 BS-3 hve the est opertos to memory rto! 35

26 D-Prllelzto *B D: p processors ler, ech processor gets full d colum slce of B, computg the relted colum slce of CB, B Commucto: N 2 p for d N*N/p*pN 2 for B Grulrty: N 3 /N 2 pn/p, B 2., B /p For : For : For : C, C,, B, Blocg oly, the colums of B! 36

27 2D-Prllelzto *B 2D: p processors squre, q:sqrtp ech proc. gets row slce of d colum slce of B computg full suloc of CB 2. Compre: S efore! B B 2. B N/q N/q Commucto: N 2 p /2 for d N 2 p /2 for B Grulrty: N 3 /2N 2 p.5 N/2p.5 For : For : For : C, C,, B, Blocg d, the colums of B d the rows of! 37

28 3D-Prllelzto *B 3D: p processors cuc, ech processor gets suloc of d suloc of B, computg prt of suloc of CB, ddtol f- to collect prts to full suloc of C. q p /3. Commucto: N 2 p /3 for d for B p*n 2 /p 2/3 p * locsze f-: N 2 p /3 the sme Grulrty: N 3 /3N 2 p /3 N/3p /3 For : For : For : C, C,, B, 38

29 Product of red d lue gves prt of grey, tht hve to e dded up to gve the full grey loc. 3D locg red, lue, reltve to lc. 39

30 3. Guss Elmto: Bsc Propertes 3. er Equtos wth dese mtrces x x x x System of ler equtos: x x x Solve Geerte smpler ler equtos mtrces. Strt wth Trsform trgulr form: 2. U

31 :

32 : U

33 We ssume tht o pvotg s ecessry smplfy or ρ > for,2,..., lgorthm: For : - For : l, / ; ed For : For : l ; ed ed ed I prctce: Iclude pvotg d clude rght hd sde. There s stll to solve trgulr system U! 4

34 5 Itermedte systems,,2,, wth d U.,,, prt of tht wll e used d chged the followg computtos.

35 6 Defe uxlry mtrces:,, l l -th colum, 2 l l l U d

36 7 Ech elmto step c e wrtte terms of the uxlry mtrces: I I I I U K : I I U wth U upper trgulr d lower trgulr. Theorem 2: d therefore U. dvtge: Every further prolem x c e reduced to Ux Solve two trgulr prolems Uxy d Uxy.

37 8 Theorem 2: d therefore U : * * * * : for I I I I I I 2 I I I I [ ] I I I I I I I I 2 2

38 3.2 Vectorzto of GE -form stdrd form: For : - For : l,, /, ; ed For : For :,, l,, ; ed ed ed Vector operto α x r SXPY rows d No GXPY U computed rowwse, columwse. 9

39 lredy computed, rems uchged, ot used ymore U ewly computed updted every step Stdrd form s lso clled rghtloog GE.

40 Frst Elmto step: Compute frst colum of Updte

41 Secod step: 2 Compute secod colum of Updte 2 2

42 Secod step: 3 Compute thrd colum of Updte 3 3

43 -st step: U Compute -th colum of Updte 4

44 Rules for dfferet,, forms: I the followg we g terchge the loops. Necessry codtos: < < Furthermore: Iermost dex,, or determes whether the computto s doe row, colum, or loc-wse. Weghts l hve to e computed efore they re used to elmte relted etres. 5

45 < -form: < For 2 : For : - l,, /, ; For :,, l,, ; ed ed ed GXPY. 6

46 lredy computed, ot used y more U lredy computed d prtlly used ewly computed uchged, ot used d U computed rowwse. Compute l,, the SXPY for st d -th row; the l,2 d so o 7

47 Frst step 8

48 Secod step 2 9

49 --st step U - 2

50 -form: < < For 2 : For 2 : l,-,- / -,- ; For : -,, l,, ; ed ew row Dot product left prt ed For : For : -,, l,, ; ed ed Dot product rght prt ed Compute l, d updte,2 ; the compute l,2 d updte,2 d,3,. ccumultg, 2

51 < < -form: For 2 : For : l,-,- / -,- ; ed For : - For :,, l,, ; ed ed ed α x r ew colum of GXPY. 22

52 eft loog GE computed, ot used U lredy computed d used uchged, ot used -, ewly computed 23

53 Frst step 24

54 Secod step 25

55 --st step U 26

56 vervew ccess to d U ccess to Computt o of U Computt o of Vector operto Vector legth row colum row colum colum colum colum row colum row row row row row colum colum colum colum row row colum colum SXPY SXPY GXPY DT GXPY DT 2/3 2/3 2/3 /3 2/3 /3 Vector legth verge of occurg vector legths 27 ptml form depeds o storge of mtrces d vector legth.