Compiling for Parallelism & Locality. Example. Announcement Need to make up November 14th lecture. Last time Data dependences and loops

Transcription

1 Complng for Parallelsm & Localty Announcement Need to make up November 14th lecture Last tme Data dependences and loops Today Fnsh data dependence analyss for loops CS553 Lecture Complng for Parallelsm & Localty 2 Sample code do = 1,6 do j = 1,5 A(,j) = A(-1,j+1)+1 j Knd of dependence: Dstance vector: Flow (1, 1) CS553 Lecture Complng for Parallelsm & Localty 3 1

2 Exercse Sample code do j = 1,5 do = 1,6 A(,j) = A(-1,j+1)+1 j Knd of dependence: Dstance vector: Ant (1, -1) CS553 Lecture Complng for Parallelsm & Localty 4 Drecton Vector Defnton A drecton vector serves the same purpose as a dstance vector when less precson s requred or avalable Element of a drecton vector s <, >, or = based on whether the source of the dependence precedes, follows or s n the same teraton as the target n loop do = 1,5 do j = 1,6 A(j,) = A(j-1,-1)+1 Drecton vector: Dstance vector: (<,<) (1,1) CS553 Lecture Complng for Parallelsm & Localty 5 j 2

3 Dstance Vectors: Legalty Defnton A dependence vector, v, s lexcographcally nonnegatve when the leftmost entry n v s postve or all elements of v are zero Yes: (0,0,0), (0,1), (0,2,-2) No: (-1), (0,-2), (0,-1,1) A dependence vector s legal when t s lexcographcally nonnegatve (assumng that ndces ncrease as we terate) Why are lexcographcally negatve dstance vectors llegal? What are legal drecton vectors? CS553 Lecture Complng for Parallelsm & Localty 6 Loop-Carred Dependences Defnton A dependence D=(d 1,...d n ) s carred at loop level f d s the frst nonzero element of D do = 1,6 do j = 1,6 A(,j) = B(-1,j)+1 B(,j) = A(,j-1)*2 Dstance vectors: (1,0) for accesses to A (0,1) for accesses to B Loop-carred dependences The loop carres dependence due to A The j loop carres dependence due to B CS553 Lecture Complng for Parallelsm & Localty 7 3

4 Parallelzaton Idea Each teraton of a loop may be executed n parallel f t carres no dependences do = 1,6 do j = 1,5 A(,j) = B(-1,j-1)+1 B(,j) = A(,j-1)*2 Parallelze loop? Dstance Vectors: (1,0) for A (flow) (1,1) for B (flow) CS553 Lecture Complng for Parallelsm & Localty 8 j Iteraton Space Parallelzaton Idea Each teraton of a loop may be executed n parallel f t carres no dependences do = 1,6 do j = 1,5 A(,j) = B(-1,j-1)+1 B(,j) = A(,j-1)*2 Parallelze j loop? Dstance Vectors: (1,0) for A (flow) (1,1) for B (flow) CS553 Lecture Complng for Parallelsm & Localty 9 j Iteraton Space 4

5 Scalar Expanson: Motvaton Problem Loop-carred dependences nhbt parallelsm Scalar references result n loop-carred dependences do = 1,6 t = A() + B() C() = t + 1/t Can ths loop be parallelzed? What knd of dependences are these? CS553 Lecture Complng for Parallelsm & Localty 10 No. Ant dependences. Conventon for these sldes: Arrays start wth upper case letters, scalars do not Scalar Expanson Idea Elmnate false dependences by ntroducng extra storage do = 1,6 T() = A() + B() C() = T() + 1/T() Can ths loop be parallelzed? Dsadvantages? CS553 Lecture Complng for Parallelsm & Localty 11 5

6 Scalar Expanson Detals Restrctons The loop must be a countable loop.e. The loop trp count must be ndependent of the body of the loop There can not be loop-carred flow dependences due to the scalar The expanded scalar must have no upward exposed uses n the loop do = 1,6 prnt(t) t = A() + B() C() = t + 1/t Nested loops may requre much more storage When the scalar s lve after the loop, we must move the correct array value nto the scalar CS553 Lecture Complng for Parallelsm & Localty 12 2: Parallelzaton (reprse) Why can t ths loop be parallelzed? do = 1,100 A() = A(-1) Dstance Vector: (1) Why can ths loop be parallelzed? do = 1,100 A() = A() Dstance Vector: (0) CS553 Lecture Complng for Parallelsm & Localty 13 6

7 1: Loop Permutaton (reprse) Sample code do j = 1,6 do = 1,5 A(j,) = A(j,)+1 do = 1,5 do j = 1,6 A(j,) = A(j,)+1 Why s ths legal? No loop-carred dependences, so we can arbtrarly change order of teraton executon CS553 Lecture Complng for Parallelsm & Localty 14 Dependence Testng Consder the followng code do = 1,5 A(3*+2) = A(2*+1)+1 Queston How do we determne whether one array reference depends on another across teratons of an teraton space? CS553 Lecture Complng for Parallelsm & Localty 15 7

8 Dependence Testng n General General code do 1 = l 1,h 1... do n = l n,h n A(f( 1,..., n )) =... A(g( 1,..., n ))... There exsts a dependence between teratons I=( 1,..., n ) and J=(j 1,..., j n ) when f(i) = g(j) (l 1,...l n ) < I,J < (h 1,...,h n ) CS553 Lecture Complng for Parallelsm & Localty 16 Algorthms for Solvng the Dependence Problem Heurstcs GCD test (Banerjee76,Towle76): determnes whether nteger soluton s possble, no bounds checkng Banerjee test (Banerjee 79): checks real bounds I-Test (Kong et al. 90): nteger soluton n real bounds Lambda test (L et al. 90): all dmensons smultaneously Delta test (Goff et al. 91): pattern matches for effcency Power test (Wolfe et al. 92): extended GCD and Fourer Motzkn combnaton Use some form of Fourer-Motzkn elmnaton for ntegers Parametrc Integer Programmng (Feautrer91) Omega test (Pugh92) CS553 Lecture Complng for Parallelsm & Localty 17 8

9 Dependence Testng: Smple Case Sample code do = l,h A(a*+c 1 ) =... A(a*+c 2 ) Dependence? a* 1 +c 1 = a* 2 +c 2, or a* 1 a* 2 = c 2 -c 1 Soluton exsts f a dvdes c 2 -c 1 CS553 Lecture Complng for Parallelsm & Localty 18 Code do = l,h A(2*+2) = A(2*-2)+1 1 Dependence? 2* 1 2* 2 = -2 2 = -4 (yes, 2 dvdes -4) 2 Knd of dependence? Ant? 2 + d = 1 d = -2 Flow? 1 + d = 2 d = 2 CS553 Lecture Complng for Parallelsm & Localty 19 9

10 GCD Test Idea Generalze test to lnear functons of terators Code do = l,h do j = l j,h j A(a 1 * + a 2 *j + a 0 ) =... A(b 1 * + b 2 *j + b 0 )... Agan a 1 * 1 - b 1 * 2 + a 2 *j 1 b 2 *j 2 = b 0 a 0 Soluton exsts f gcd(a 1,a 2,b 1,b 2 ) dvdes b 0 a 0 CS553 Lecture Complng for Parallelsm & Localty 20 Code do = l,h do j = l j,h j A(4* + 2*j + 1) =... A(6* + 2*j + 4)... gcd(4,-6,2,-2) = 2 Does 2 dvde 4-1? CS553 Lecture Complng for Parallelsm & Localty 21 10

11 Concepts Improve performance by... mprovng data localty parallzng the computaton Data Dependences teraton space dstance vectors and drecton vectors loop carred Transformaton legalty must respect data dependences scalar expanson as a technque to remove ant and output dependences Data Dependence Testng general formulaton of the problem GCD test CS553 Lecture Complng for Parallelsm & Localty 22 Next Tme Lecture Value dependence analyss CS553 Lecture Complng for Parallelsm & Localty 23 11