Design of Parallel & tlrgh Performance Computing. about Performance I. Reasoning. Principles. Balance. Roofline Mode -

Size: px

Start display at page:

Download "Design of Parallel & tlrgh Performance Computing. about Performance I. Reasoning. Principles. Balance. Roofline Mode -"

Justin Byrd
5 years ago
Views:

1 Balance Design of Parallel & tlrgh Performance Computing about Performance I Reasoning Roofline Mode Principles

2 vector matrix Roofline Model ( Williams et al ) resources in a microarchitecture associated! features Sanelarelth memory B esytyleyche ] deter that found performance program peak performance F tfbnskycle ] work W [ flops ] transferred cache a men Q Estes ] a for given problem min W = work Hime coupled Ly min Q = 110 aplenty Operational intensity I Given a program empty Cade lnturtdou higs I a compute Sound low I > memory sound assume Icu ) Wcw ) QCu ) Example vector Sun y=xty OCD asymptotic founds matrix Icu ) product y=a OG ) on I fast Fourier transform Ocloju ) matrix product C=AB Ocu )

3 Operational Intensity Example Mahir Multiplication Assumptions cache cache 86ek=8 doubles siugkn leade flops We want to estimate Q W=2u3 1 / a B Ca AD A 1 2ns o o to a a 1) Triple loop 2) Blocked 8lb " ( tf # *% entry of C ut8n doubles 1 Slack of C ' 35 ' Er B AD doubles 2 end > ofc same 2 sleek of C same total Sis doubles total 2ns o o o 8 ftp2yddoushs Ica ) = OC i ) bzff )

4 i T Roofline Rodell Williams et al Computer Program men I= W/Q I fbpslsyde ] bandwidth A LLC tsyteskyck ] T = runtime I cycles ] CPU peak perf tfbpskyclei P= WIT ( performance ) [ fbpskyck ] Roofline plot ( example T= 2 A=4 ) 1 Ptfbpskyde ] sound jasedenp fp±pi ) reason =Ww a= 2f ^ Sind 2 r Sounds ( PET ) PZ p I logp a legit leg B ' k I program Isl P 2 A run on Some I input guts ^ > YYYZ 1 2 I I f6ps/j > tif

5 ' Ppenatroud lnhhtidy Upper Lonely daxpy=a ty asymptotic sest exact 0 ( 1 ) OCI ) 2 1/12 Icu ) = War )/Q(u ) y length n It flops / # Sites memo eaehe A BC= nxn a scalar dgemv y Axty OCI ) 04 ) 11/4 fft OCGSU ) Occogy ) dgemm C=ABtC ocn ) o ( of ) E YI should fe y=eade till sharp 3u2 if 828 Example y= KMB =) n E 700 back to slides

6 ) G Balance Principles I Ckuey 86 ) Compete Pegram men Qy data transfer men a LLC s LLC sire j W work = # ops CPU IT The computer is called Salaam if ire compute time = data transfer the assuming perfect utilization = assume it at increases Ese Ip= ; r Jr how A nesalauee? a) A a A set A grows slower than 5! s y x 8 what is?

7 Example I Mahn multiplicities Ca ABTC algorithm worth opkmel I= hot QCTF ) Ip = =o( of ) tat r e?g Example 2 FFT / dorky algorithm ark optimal logo ) Is = hofy@c1gpitsaur ya pots are unnealrttfe

8 tuned there Lou 7 Balance principles I ( Czechowski et al ) coal none detailed principles multi for ones algorithm / architecture adesrgn assess web of HW tends Computer ; Algorithm slow men PRAM W work Di depth P fast men di lately B throughput sing dnansaehon fired Qp each poll IT perf Pp processors Processor is faleuafd ;f Them E Tamp ( " compute men transfers of she A assumptions optimal W W/Q two * set Qp rn fun Qnx? are general fends and some Arnet results " as of a put > F 3 a

9 par Derivation principles 1) ESH mete Tmen Idea did DAG in levels level I o o o a o o q M 4 \o/ levels qz men parsley of tie t level D= o a a 0 9D In each level a A model a +9 8 ' ' ' Tuey X g ( at 9 ) = a D +0 Qt Qpinx 2) Estimably Tap Brent 's theorem Taya CD+ men pan amputee Balance Them a Tamp qn < r a+oyt laa ei+ ) ^ T th men parallelism parallelism algorithm Gupute algorithm

10 2004 Example 1 ; matrix multiplication use Q >cr[ Fp Chou > et al ) Icu ) = 0 ( ITF) stance pwuehple Fr Cp PIE OC P ) it a it resdauee A on ap p " a p p ( A as p ay) and Example 2 sending # FT PF zocigrg ) on a2j Pg ) Back to slrdy

Lecture 23: Illusiveness of Parallel Performance. James C. Hoe Department of ECE Carnegie Mellon University

18 447 Lecture 23: Illusiveness of Parallel Performance James C. Hoe Department of ECE Carnegie Mellon University 18 447 S18 L23 S1, James C. Hoe, CMU/ECE/CALCM, 2018 Your goal today Housekeeping peel