register allocation instruction selection Code Low-level intermediate Representation Back-end Assembly array references converted into low level operations, loops converted to control flow Middle2 Low-level Intermediate Representation conventional optimizations loops,array references are preserved Middle1 loop-level transformations High-level Intermediate Representation Source Program Front-end syntax analysis type-checking symbol table Organization of a Modern Compiler
"! $ &,.- / 01 23 4 / 01 3 4 9 3 9 =< > 2 @? A 2 CB D@EFG IHE J KML N IOE PE J Q Q DSRT Q Q D N IOE NU VW U H W O QC XE OO U CN E N I IXIO E C E N I O Q ZHH F V U I W NU O QC Z[E I N Q I X JJ J Q D OQ Q G J Q Q D N IOE / 01 \ 3 4 ] \ \ 3 ^ _ 1 ` ] \ \ / 01 a 3 \< 4 ] a \ 3 ] a \ b ] \ \ / 01 a 3 \< 4 / 01 c 3 \< 4 a ] a c ed 3 ] a \? ] c \ f
GF Q ZH OG Q F J IOEN PE h Z g N QVR D@EFG IHE K L J Q D OQ
i j k l & m "! - j n & & - j & & m! o p, rq vuts s r xw y z v t{ p q v { x } t ~ w v y r y
Š ŒŒŒ o &, l & $ ˆ Š ˆ. Š ˆ. Š ˆ Š ˆ. Š Œ ŒŒ Ž ˆ. oƒ 1 N I S 1 DO I = 1, N DO J = 1,M M J!! - ƒ!
$ 1 DO K = 1, M DO L = 1, N SK,L N L DO I = 1, N DO J = 1,M SI,J 1 M J 1 M 0 1 1 0 I J 1 N I = K K L
$ $,,, -
, Question How do we determine when loop permutation is legal? Transformed loop will produce different values A[3,1] for example => permutation is illegal for this loop. A[I,J] = A[I-1,J1] 1 DO J = 1, M DO I = 2, N After loop permutation Assume that array has 1 s stored everywhere before loop begins. DO I = 2, N DO J = 1, M A[I,J] = A[I-1,J1] 1 N 1 I 2 N J,
,,! l oƒ, l m š œ. ž Ÿ š ž Ÿ œ ž, o & &, -
Question How do we generate loop bounds for transformed loop nest? 1 K 1 N L DO K = 1, N DO L = I, N N SK,L 0 1 1 0 I J = K L 1 N I DO I = 1, N DO J = 1, I SI,J 1 N J
i p & p &,, l p &, - ª, l l f
«h
-, - $ $, i
Most problems regarding correctness of transformations and code generation can be reduced to these problems. ii Enumerate all integer solutions. i Are there integer solutions? x is an n vector of unknowns, b is an m vector of integers, where A is a m X n matrix of integers, Given a system of linear inequalities A x < b Two problems
Region described by inequality is convex if two points are in region, all points in between them are in region 3x 4y <= 12 x 3x4y = 12 y Inequality half-plane 2D, half-space>2d Equality line 2D, plane 3D, hyperplane > 3D Intuition about systems of linear inequalities
Region described by inequalities is a convex polyhedron if two points are in region, all points in between them are in region 3x - 3y <= 9 x x >= - 3x4y <= 12 y <= 4 y Conjunction of inequalties = intersection of half-spaces => some convex region Intuition about systems of linear inequalities
$ $, - - n &, - k $ m -
- i S1 executes before S2 ii S1 and S2 both read from the same location Input dependence S1 -> S2 i S1 executes before S2 ii S1 and S2 write to the same location Output dependence S1 -> S2 i S1 executes before S2 ii S1 reads from a location that is overwritten later by S2 output Anti-dependence S1 -> S2 x = 2 y = x 1 x = 3 y = output anti flow i S1 executes before S2 in program order ii S1 writes into a location that is read by S2 Flow dependence S1 -> S2 control Dependences data flow anti output
Key notion Aliasing two program names may refer to the same location like Xi and Xj May-dependence vs must-dependence More precise analysis may eliminate may-dependences => Unless we know from interprocedural analysis that the parameters i and j are always distinct, we must play it safe and insert the dependence. Answer If i = j, there is a dependence otherwise, not. Question Is there an output dependence from the first assignment to the second? procedure f X,i,j begin Xi = 10 Xj = end Example When you are not sure whether a dependence exists, you must assume it does. - Real programs imprecise information => need for safe approximation Conservative Approximation
How do we compute dependences between iterations of a loop nest? Iteration I1,J1 is said to be dependent on iteration I2,J2 if a dynamic instance I1,J1 of a statement in loop body is dependent on a dynamic instance I2,J2 of a statement in the loop body. Dependence between iterations Dynamic instance of a statement Execution of a statement for given loop index values each I,J value of loop indices corresponds to one point in picture I DO I = 1, 100 DO J = 1, 100 S J Loop level Analysis granularity is a loop iteration
± ² ³ ² µ ² ± µ š ž ¹ œ ž Ÿ º»¼»œ ½ ½ ž ¹ º»»œ ½ ½ p À & Á  Á à p Ä Á Â Ä Á Ã Ä Ž ÅÇÆur ts t ~ uw ut vt È É p Ê Á Â Ë Á à Šy v Ì vt vuu ÍÎ w { v s r xw É p! Á Ï Á Ð p Ä Á ÏÒÑ Á Ð Ä Ž Å u vt È t ~ uw t ruæ ts É p Ê Á Ð Ë Á Ï Å y v Ì vt vuu ÍÎ w { v s r xw É p Á  Á  p Ä Á Â Ñ Á Â Ä Ž ÅÇÆur rts x z u } uwv Ì uw utè É p Ê Á Â Ê Á  Šy v Ì vt vuu Í Î w { v s r xw É f
í ä ä î è æè é ä ä ì è æè ä ä è æè Ú Û Ú Û Û Ú Û Ú Û Û Š Á  Á à à ½ ½ ½ ž Ÿ ± ³ ðï ã Ü å ß æëê ã Ü å ß ã Ü å ß æëê ã Ü å ß ã Ü å ß æ ç ã Ü å ß ß â Û à Þ Þ Ü â Ú Þ Þ ß Ý Ûáà Þ Þ Ü Ý Ú Þ Þ Ù o - À & ˆØ  ˆ ž ¹ º»Õ»œ Ÿ ½ Ÿ Ö»œ Ÿ º»¼»œ Ÿ Ÿ»œ Ÿ ½ ½ š ž ¹ ž Ÿ Ô ¹ ¹ š ž ¹ œ ž ¹ ¹ ² ³ ² µ ² ² Óµ ± µ
ø ö ù óø ï òñ óô õò ó úö ù úò öúò üúû ûõò úý öúþ ò ú ö õò õÿ ø ü ÿ
ï
! " $ &, - &...., / -... 0 1 2 2 3 4 9 < ê < = >? û? ô ûõò úý >@ ô óóúò õ ü BA < < = < = < ï C
0D BA < < = < = < ûõò úý úe ù @ üüú ú ö òô úó? F @ G H I KJ õ ü F ÿÿ ü> L L L L L L L A A < A < A < < M M M M M M M N O O O O O O O P A A A Q R R R R R R R S ï T
! U " $ & " $ V & V - &.. XW V/ -... 0 1 2 2 3 4 Z [ \ ] Z [ \ZB^_ a`i b c d egf a` b _h \ A i = i j ò kú@ óÿ úe ô û @ õ ù? ô û @ öú@ ] òô ó òô úó ú@ ú l ø õÿ ô üúóô m ôò ú l ø õÿ ô üúóô ï n
oqp r sut r vxw o p y s t y v z p r { p y } oop r p y vq~ o t r { t y vv 2 0 2 z 9 BA i = i ƒ i 9 BA i = i 0 0 ˆ
0 4 " $ & " $ V & V - &.. XW V/ -... Z [ \ ] Z [ \ZB^_ a`i b c d egf a` b _h \ BA i = i ˆŠ
0 Œ Ž Ž š š œ š ž šÿ š Ÿœ š š š Z \ < Z \ Z \ < Z \ j ª k «[ < ª G ª ± ² F ² ª ³ ² ˆµ
ˆ ˆ For a given I, the J co-ordinate of a point in the iteration space of the loop nest satisfies maxl1i,l2i <= J <= minu1i,u2i I U1 L1 L2 U2 J general polyhedral iteration spaces! Min s and max s in loop bounds mayseem weird, but actually they describe
¹ º» ¼ ½» ¹ ¼ º ½ ½»¹ ¹ ¹ ¼ À º º Á 2 À¼ Ž àŽ ÃÂ Ä Å ª ³ ² Æ«@ ª Ç ª? È? ³ ª > ² ɲ ² > ² ²³ Ê ª G ÌË ÌË A Æ? ³ ³ l ³ k Å ² ª ª ²³ È F³ ª? @ ³ Å ª ³ ² @F ² Ê k Å F Ç Ç ª «F³ >? k G @ ²³ @F G ª ³ ² ª ª? @ ² È F Ç Í > ² ų Ê Î Î ³ ª Î Î ³ ª ³ ² Å ²³ l Å ³ ª ³ ˆÏ
Ð À Ñ» ¹ ¹¼ ¼ ¹ º ¼ ½¼ ¹» 2» ¼ 2» ¹»»¹ ¼ º ½» Ñ ¼» º ½ ¼ º» 0» 0 ¼ ¹ ¼» À»¹ ¼» Ñ» ~ Ò ÓKÔ 4 Õ 2Ö 2Ö Ö» 0 ¹ ½ ¹ ¹ Ö¼ º Ö 4 ؈
» 0 ¼ ¹ ¼» Ö À»¹ Ö¼» Ö Ñ» ~ Ò ÓKÔ 4» Ö À»¹ Ö¼» ½0 ¹¼ z À! Ö À ¹ ÚÙ Ö»ÛÜ ¹ ¼ ¹ ¹ ¼»¹ Ö¼ ¼»À¹»¹ Ö¼ ŒÝ Þ À ¹ ¼ ¹ ¹ ¼»¹ Ö¼ Ö ß ¹ ¼ z À»¹ Ý Ö Ö ¼» ½0 ¹¼ z À ¹» à À» Ü ¼ 2Ö ¼ Ö À»¹ Ö¼ z À¹»¹ º Ö¼ ¼»¹ ¹ ¼» 0 ¼ À Ö ¹ß ¼ z À ¹»¹ ¼á Ñ» 0 ¹ ¼ Ñ Ö ½À ¼ Ö» Ö ½À» 0 ¼ À Ö ¹ß ¼ z À ¹»¹ ¹ Ö Ö â z À ¹»¹ 0 Ö À ¼ Ö» ½ Ö¼» ¹ ¼ Ñ ¹ ¼» Ö¹ ¼ z À ¹»¹ À» 0 ¼ º» Ñ ˆ
ˆ C 2x 3 y = x <= y <= -9 - equations & inequalities 2x 3 y z = 3x 4 y = 3 - several equations, several variables 2 x 3 y = - one equation, several variables Presentation sequence Diophatine equations use integer Gaussian elimination Solve equalities first then use Fourier-Motzkin elimination
ˆ T Intuition Think of underdetermined systems of eqns over reals. Caution Integer constraint => Diophantine system may have no solns Solutions z = t => x = 3t - 3 y = 3-2t y = 3-2t Rewrite equation as 2z y = 3 GCD2,3 = 1 which divides 3. Let z = x floor3/2 y = x y 4 2x 3y = 3 GCD2,1 = 1 which divides 3. Solutions x = t, y = 3-2t 3 2x y = 3 2 2x = One solution x = 3 1 2x = 3 No solutions Examples One equation, many variables Thm The linear Diophatine equation a1 x1 a2 x2... an xn = c has integer solutions iff gcda1,a2,...,an divides c.
ˆ n If a1 is the smallest co-efficient in 1, we are left with 1 variable base case. Otherwise, the size of the smallest co-efficient has decreased, so we have made progress in the induction. Observe that 1 has integer solutions iff original equation does too. Now gcda,b = gcda mod b, b => gcda1,a2,...,an = gcda1, a2 mod a1,..,an mod a1 => gcda1, a2 mod a1,..,an mod a1 divides c. where we assume that all terms with zero coefficient have been deleted. a1 t a2 mod a1 x2... an mod a1 xn = c 1 Suppose smallest coefficient is a1, and let t = x1 floora2/a1 x2... flooran/a1 xn In terms of this variable, the equation can be rewritten as If of variables = 1, then equation is a1 x1 = c which has integer solutions if a1 divides c. If smallest coefficient = 1, then gcda1,a2,...,an = 1 which divides c. Wlog, assume that a1 = 1, and observe that the equation has solutions of the form c - a2 t2 - a3 t3 -...-an tn, t2, t3,...tn. Inductive case Base case Proof WLOG, assume that all coefficients a1,a2,...an are positive. We prove only the IF case by induction, the proof in the other direction is trivial. Induction is on minsmallest coefficient, number of variables. Thm The linear Diophatine equation a1 x1 a2 x2... an xn = c has integer solutions iff gcda1,a2,...,an divides c.
Ï = Does gcda1,a2,...,an divide c? - Does this have integer solutions? Eqn a1 x1 a2 x2... an xn = c Summary
Ï Š For a matrix with a single row, column echelon form is x 0 0 0...0 Key concept column echelon form - "lower triangular form for underdetermined systems" looks lower triangular, right? Solution is a = 4, b = t 2 0a b T = It is hard to read off solution from this, but for special matrices, it is easy. T 3 x y z = We can write single equation as It is useful to consider solution process in matrix-theoretic terms.
3x y z = Substitution t = x y 2z New equation 3t 2y 2z = Substitution u = yzt New equation 2u t = Solution u = p1 t = -2p1 Backsubstitution y = p2 t = -2p1 z = 3p1-p2- Backsubstitution x = 1-p1p2 y = p2 z = 3p1-p2- Ï µ Solution to original system 12-a-b U1U2U3 a b T -3a-b b Product of matrices = 2 - -1-1 3-1 0 0 1 = 1 0 0 T Solution a b U1U2U3 U3 1 2 0 1-2 0 0 1 0 0 0 1 = 1 2 0 3 2 2 1 0 0-1 1-1 0 0 1 U2 = 3 2 2 3 1-1 -2 0 1 0 0 0 1 U1 3
Ï ˆ INTEGER GAUSSIAN ELIMINATION Question Can we convert general integer matrix into equivalent lower triangular system? z z = arbitrary integer -2 0 1 0 0 y = => y = 3 x x = Easy special case lower triangular matrix It is not easy to determine if this Diophatine system has solutions. z Example 2x 3y 4z = => x - y 2z = 2 3 4 1-1 2 y x = Key idea use integer Gaussian elimination Systems of Diophatine Equations
Ï Ï AU1U2...Ukx = b => AU1U2...Ukx = b => x = U1U2...Ukx Proof AU1U2...Uk is lower triangular say L Solve Lx = b easy Compute x = U1U2...Ukx Overall strategy Given Ax = b Find matrices U1, U2,...Uk such that - Use row/column operations to get matrix into triangular form - For us, column operations are more important because we usually have more unknowns than equations Integer gaussian Elimination
Ï Ø Question Can we stay purely in the integer domain? One solution Use only unimodular column operations using rationals. of original system requires solving lower triangular system Intuition With some column operations, recovering solution -4 y 2 0 x = which has no integer solutions! 1 0 2 1-3 2 3 x y = Solution x = -, y = 1 Caution Not all column operations preserve integer solutions.
Ï ã 1-1 0 1 2 3 n = -1 2 1 1 x = x n y y = y Check c Add an integer multiple of one column to another 1 0 0-1 2 3 2-3 - b Negate a column Check x = x, y = - y 0 1 1 0 x = y, y = x 2 3 3 2 Let x,y satisfy second eqn. Let x,y satisfy first eqn. a Interchange two columns Check Unimodular Column Operations
Ï ä 1 0 0-2 0 x x = = => => y y = 3 z z = t z 0 0 1 t y = 1-2 0 3 x -1 3-2 4-2t = -1 t 0 0 1 0 0 1 0 0 1 0 0 1 0 1 0 1 0-2 1-1 0 1 0 0 0 1 0 0 1 0-2 1 0 1 0 0 2 3 4 2 3 0 2 1 0 0 1 0 1 0 0 => => => => 1-1 2 1-1 0 1-2 0-2 0-2 0 z 2 3 4 1-1 2 y x = Example
Ï å 4. The product of two unimodular matrices is also unimodular. of 1 or -1. 3. A unimodular matrix has integer entries and a determinant 2. Unimodular column operations can be used to reduce a matrix A into lower triangular form. on the matrix A of the system A x = b preserve integer solutions, as do sequences of these operations. - interchanging two columns - negating a column - adding an integer multiple of one column to another 1. The three unimodular column operations Facts
Ï æ
çø Note Even in regular Gaussian elimination, we want column echelon form rather than lower triangular form when we have under-determined systems. x x x x 0 0 x 0 0 is lower triangular but not column echelon. Point writing down the solution for this system requires additional work with the last equation 1 equation, 2 variables. This work is precisely what is required to produce the column echelon form. ii column j1 is zero if column j is. Column echelon form Let rj be the row containing the first non-zero in column j. i rj1 > rj if column j is not entirely zero. Detail Instead of lower triangular matrix, you should to compute column echelon form of matrix. x = U x where x is the solution of the system Lx = b 3. If explicit form of solutions is desired, let U be the product of unimodular matrices corresponding to the column operations. 2. If Lx = b has integer solutions, so does the original system. 1. Use unimodular column operations to reduce matrix A to lower triangular form L. Algorithm Given a system of Diophantine equations Ax = b