Numercal Propertes of the LLL Algorthm Frankln T. Luk a and Sanzheng Qao b a Department of Mathematcs, Hong Kong Baptst Unversty, Kowloon Tong, Hong Kong b Dept. of Computng and Software, McMaster Unv., Hamlton, Ontaro L8S 4L7, Canada ABSTRACT The LLL algorthm s wdely used to solve the nteger least squares problems that arse n many engneerng applcatons. As most practtoners dd not understand how the LLL algorthm works, they avoded the ssue by referrng to the method as an nteger Gram Schmdt approach (wthout explanng what they mean by ths term). Luk and Tracy 1 were frst to descrbe the behavor of the LLL algorthm, and they presented a new numercal mplementaton that should be more robust than the orgnal LLL scheme. In ths paper, we compare the numercal propertes of the two dfferent LLL mplementatons. Keywords: LLL algorthm, unmodular transformaton, QR decomposton, reduced bass, Gauss transformaton, plane reflecton, numercal overflow and underflow. 1. INTRODUCTION The famous algorthm due to Lenstra, Lenstra and Lovasz 2 has many mportant applcatons; for example, wreless communcaton, cryptography, and GPS (see Hassb and Vkalo 3 and references theren). In some of these applcatons, researchers use the LLL algorthm as a precondtoner n solvng an nteger least squares problem. Although the LLL algorthm s often referred to as an nteger Gram-Schmdt procedure, no one has explaned the workngs of such a process. Luk and Tracy 1 acheved a breakthrough by showng how an LLL reducton can be mplemented usng orthogonal nstead of Gauss transformatons. The purpose of ths paper s to compare the two dfferent numercal mplementatons of the LLL method. Ths paper s organzed as follows. In Sectons 2 and 3, we descrbe the orgnal 2 and new 1 mplementatons of the LLL algorthm. In Secton 4, we present the result 1 that the two dfferent mplementatons gve the same answers n exact arthmetc. Lastly, n Secton 5, we conclude the paper by presentng examples to compare the numercal propertes of the two mplementatons. 2. LLL ALGORITHM Gven a nonsngular matrx B R n n, an dea n Lenstra et al. 2 s to construct a unmodular matrx M Z n n so that the columns of BM become almost orthogonal; a usual consequence s that the condton number of BM wll become much smaller than that of B. Defnton 1. A nonsngular matrx M s unmodular f det(m) = ±1. Lemma 1. A nonsngular nteger matrx M s unmodular f and only f M 1 s an nteger matrx. A key concept n the LLL paper 2 s that of a reduced bass. Consder the QR decomposton of B: where Q R n n s orthogonal, D dag(d ) R n n s dagonal wth Q T B = DU, (1) d > 0, for = 1, 2,..., n, and U (u,j ) R n n s upper trangular wth ones on ts dagonal: Send correspondence to S. Qao: qao@mcmaster.ca u, = 1, for = 1, 2,..., n.
Defnton 2. The columns of B form a reduced bass f and where 0.25 < ω < 1 s a parameter that controls the rate of convergence. u,j 0.5, for 1 < j n, (2) d 2 (ω u2 1,j )d2 1, for 2 n, (3) Condton (2) states that the absolute value of any strctly upper trangular element of U s at most 0.5. Condton (3) states that the dagonal elements of U must be ordered n a certan manner. Lemma 2. Snce the value of the quantty nsde the parentheses n (3) s always less than one, an upper trangular matrx B R n n wth a constant dagonal satsfes condton (3). Example 1. The columns of ths trangular matrx B R n n form a reduced bass: 1 0.5 0.5 0.5 1 0.5 0.5. B = 1.. 0.5..... (4)... 1 0.5 1 The matrx s very ll-condtoned, for Luk and Tracy 1 show that ts condton number ncreases lke (1.5) n 2 /2. Let us descrbe the actons of the LLL algorthm by showng how condtons (2) and (3) are enforced. Condton (2) s easy to mpose on U (u,j ), an upper trangular matrx wth a unt dagonal. We begn by defnng elementary unmodular transformaton. Let < j, and let e Z n and e j Z n denote the unt coordnate vectors n the -th and j-th drectons, respectvely. Defne M j Z n n by where γ s an nteger. M j I γe e T j, (5) Lemma 3. The matrx M j defned n (5) s an nteger unmodular transformaton. We use M j to ensure that the (, j)-th element of U s suffcently small. Suppose that (2) s not satsfed for some and j; that s, u,j > 0.5. Calculate γ as the nteger closest to u,j : γ = u,j. (6) Construct the unmodular matrx M j wth ts (, j)-th element equal to γ. Apply M j to B and to U: B BM j and U UM j. (7) The (, j)-th element of the new U satsfes (2). We summarze the actons to enforce (2) n the next procedure. PROCEDURE DECREASE(, j) Gven B and U, calculate M j and γ usng (5) and (6), respectvely. Apply M j to B and U: B BM j and U UM j. Notaton 1. The matrx Π Z n n denotes a permutaton n the ( 1, ) plane, where 2 n. Notaton 2. The matrx X R n n denotes a transformaton n the ( 1, ) plane, where 2 n. It has the form: I 2 X µ 1 ξµ 1 ξ. (8) I n
For condton (3), we use the two numercal transformatons defned n the two notatons. Note that and that X 1 s gven by X 1 = det(x ) = 1, (9) 2 ξ 1 ξµ 1 µ I n. (10) The matrx X 1 s made up of a product of two Gauss transformatons; here s a quck llustraton: [ [ [ ξ 1 ξµ 1 ξ 0 1 =. 1 µ 0 1 1 µ Ths matrx X 1 s a workhorse n the LLL algorthm, and the followng relaton s key: [ [ [ [ ξ 1 ξµ 1 µ 0 1 1 ξ = 1 µ 0 1 1 0 0 1. (11) In words, equaton (11) says that the matrx X 1 restores the trangularty of a permuted trangular matrx. Note that both upper trangular matrces n (11) have ones on ther dagonals. Suppose that the (3) s not satsfed for some : d 2 < (ω u2 1, )d2 1. We nterchange columns and 1 of B and those of U: We then use the transformaton X 1 B BΠ and U UΠ. (12) of (10) to restore U to trangular form: U X 1 U. (13) Lenstra et al. 2 gve the formulas that are used to update the squares of the dagonal elements d 1 and d of D. Specfcally, ˆd 2 1 = d2 + µ2 d 2 1 and ˆd2 = (d 2 d2 1 )/ ˆd 2 1, (14) where ˆd 1 and ˆd are the new dagonal elements. The paper 2 also gves the values of ξ and µ n (8). As s obvous from (11), µ s gven by µ = u 1,. (15) In addton, ξ s gven by The actons to enforce (3) are wrtten out n the next procedure. ξ = µ d 2 1 /(d2 + µ2 d 2 1 ). (16) PROCEDURE SWAP() Gven D 2, B, and U, update D 2, swap columns 1 and and those of B and of U, and use the transformaton X 1 to transform U back to trangular form: D 2 D 2 new, B BΠ, and U X 1 UΠ. (17) The matrx Dnew 2 1 s obtaned by (14) and X s computed by the equatons (10), (15), and (16). Luk and Tracy 1 use the two procedures, Decrease and Swap, to construct an algorthmc descrpton of the LLL algorthm. The orgnal LLL paper 2 contans a proof of convergence, but not an algorthmc descrpton, of the method. It s far to say that the algorthmc descrpton nspred Luk and Tracy 1 to derve ther new mplementaton. Although the LLL algorthm has shown to be an effectve tool 3 to reduce the condton number of most ll-condtoned matrces that occur n practce, t does not modfy the ll-condtoned matrx B of (4) because ts columns already form a reduced bass.
ALGORITHM LLL Gven B, transform ts columns so that they wll form a reduced bass. compute the QR decomposton of B to get D 2 and U; set k 2; whle k n f u k 1,k > 0.5 then DECREASE(k 1, k); f d 2 k < (ω u2 k 1,k )d2 k 1 then SWAP(k); k max(k 1, 2); else for = k 2 down to 1 f u,k > 0.5 then DECREASE(, k); k k + 1. 3. A NEW IMPLEMENTATION Luk and Tracy 1 extend the dea of a reduced bass formed by column vectors to that of a reduced trangular matrx. Let B R n n be nonsngular. Consder ts QR decomposton: Q T B = R, (18) where Q R n n s orthogonal and R (r,j ) R n n s upper trangular wth a postve dagonal: r, > 0, for = 1, 2,..., n. Ths extenson 1 leads to a new algorthm to transform a gven matrx B to a reduced trangular matrx R. Defnton 3. The columns of B form a reduced bass f and r, 2 r,j, for 1 < j n, (19) r 2, [ω (r 1, /r 1, 1 ) 2 r 2 1, 1, for 2 n, (20) where 0.25 < ω < 1 s a parameter that controls the rate of convergence. Defnton 4. An upper trangular matrx R s reduced f ts elements satsfy the condtons (19) and (20). Proposton 1. Gven B R n n, the new algorthm generates an orthogonal matrx Q R n n and a unmodular matrx M Z n n to transform B nto a trangular matrx R: Q T BM = R, (21) so that R s reduced. The columns of BM form a reduced bass as defned n the LLL paper. The new approach 1 enforces condtons (19) and (20). Whle condton (19) states that any dagonal element of R s at least twce as large as any other element of R along the same row, condton (20) states that the dagonal elements of R must be ordered n a certan way. We use M j of (5) to ensure that the (, j)-th element of R s suffcently small relatve to r,. Suppose that (19) s not satsfed for some and j; that s, Calculate γ as the nteger closest to r,j /r, : r, < 2 r,j. γ = r,j /r,. (22) Construct the unmodular matrx M j wth ts (, j)-th element equal to γ. Apply M j to R: R RM j, (23)
and accumulate the transformatons n M: M MM j. It s easy to check that the (, j)-th element of the new R n (23) satsfes (19). For condton (20) we need to use a plane reflecton 4 (a basc numercal tool that s closely related to the more famlar plane rotaton). PROCEDURE NEWDECREASE(, j) Gven R and M, calculate M j and γ usng (5) and (22), respectvely, and apply M j to both R and M: R RM j and M MM j. Notaton 3. The symmetrc matrx J R n n denotes a plane reflecton n the ( 1, ) plane, where 2 n. It has the form: I 2 J c s s c, (24) where c 2 + s 2 = 1. Note that I n det(j ) = 1, (25) just as det(x ) = 1 n (9). Luk and Tracy 1 use plane reflectons nstead of plane rotatons because the X s are closely related to plane reflectons, as wll be seen n the next secton. Suppose that (20) s not satsfed for some : r 2, < [ω (r 1,/r 1, 1 ) 2 r 2 1, 1. We nterchange columns and 1 of R: and use a plane reflecton J to restore R to trangular form: We accumulate the transformatons n M and Q: R RΠ, (26) R J R. (27) M MΠ and Q QJ. Now, we have all the tools to present our new algorthm as a matrx decomposton technque. PROCEDURE NEWSWAP() Gven R, M, and Q, swap columns 1 and of R and those of M, use a plane reflecton J to transform the permuted R back to trangular form, and update Q: R J RΠ, M MΠ and Q QJ. (28) ALGORITHM NEW compute B = QR; set M I and k 2; whle k n f r k 1,k 1 < 2 r k 1,k then NEWDECREASE(k 1, k); f r 2 k,k < [ω (r k 1,k/r k 1,k 1 ) 2 r 2 k 1,k 1 then NEWSWAP(k); k max(k 1, 2); else for = k 2 down to 1 f r, < 2 r,k then NEWDECREASE(, k); k k + 1.
4. EQUIVALENCE RESULT There are many smlartes between Algorthms New and LLL. Both algorthms am to reduce the gven matrx B to a trangular form. A major dfference les n the transformatons used. Algorthm New apples plane reflectons J of (24) drectly to R, whle Algorthm LLL apples specal transformatons X 1 of (10) to U and D 2 separately. A sgnfcant result 1 s that the two transformatons are related va J = D 1 X 1 D 2, (29) where D 1 and D 2 are n n dagonal matrces. Thus, we may vew X 1 as a scaled plane reflecton. Luk and Tracy 1 show that n exact arthmetc, the two algorthms produce dentcal numercal results. Representng the effect of transformatons (26) and (27) by we wrte out the key 2-by-2 transformatons as follows: [ [ ˆα ˆγ c s = 0 ˆβ s c Defne a new transformaton Y by If we choose Y [ 1/ˆα 0 0 1/ˆβ then we get 1 Y = and [ 1 ξ 0 1 R new = J RΠ, [ α γ 0 β [ c s s c [ 0 1 1 0 [ α 0 0 β. (30). (31) ξ = ˆγ/ˆα and µ = γ/α, (32) = Y [ ξ 1 ξµ 1 µ [ 1 µ 0 1 [ 0 1 1 0. (33) Note that (33) s exactly equaton (11) for the LLL method. Also, we can easly prove that the µ and ξ as defned n (32) have the same values as the µ and ξ as defned n (15) and (16). Thus, the transformaton Y of (31) s exactly the 2 2 part of the workhorse X 1 of the LLL algorthm. Let E 1 D α 0 0 β, (34) where E 1 R ( 2) ( 2) and E 2 R (n ) (n ) are postve dagonal matrces. Defne E 1 E1 1 D 1 ˆα 0 0 ˆβ and D 2 1/α 0 0 1/β E 2 E 2 E 1 2. (35) Then J = D 1 X 1 D 2. (36) We see that D 2 reduces R to a unt-dagonal trangular matrx (namely U), and that D 1 gves the new dagonal of D 2 R after beng transformed by X 1. Therefore, we conclude that Algorthms LLL and New produce the same numercal results n exact arthmetc. It also follows that the convergence result for Algorthm LLL s applcable to Algorthm New. The former algorthm s numercally more effcent n that t avods the computaton of square roots, whch s one reason why t updates D 2 nstead of D. Thus, we may vew the transformatons n the LLL method as square-root-free plane reflectons. The potental cost for ths effcecy s a possble loss n numercal accuracy, as wll be shown n the next secton.
5. NUMERICAL PROPERTIES As ponted out n the last secton, a sgnfcant dfference between Algorthms New and LLL s that New works drectly on R whle LLL works on U and D 2 ndvdually. Put t smply, New computes r, whle LLL calculates d 2. Consequently, Algorthm LLL s susceptble to underflow (respectvely overflow) exceptons when the dagonal elements d s are small (respectvely large). For our dscusson, we assume standard IEEE floatng-pont arthmetc. In sngle precson, we would have mnmum exponent value e mn = 126 and maxmum exponent value e max = 127. Due to the presence of denormals, a number x underflows f x < 2 126 23 = 2 149, whereas the number x overflows f x 2 128. Even f the quanttes d 2 s are not small or large enough to cause exceptons, a straghtforward mplementaton of the LLL algorthm could stll result n errors. Let ω = 0.75, and consder the followng 2-by-2 upper trangular matrx [ α µ α R =. 0 0.5α The condton (20) s not satsfed when µ < 0.5. Recall the updatng formula (14): ˆd 2 = (d2 d2 1 )/ ˆd 2 1. The numerator (0.5 α 4 ) may readly underflow or overflow; for example, n sngle precson, an underflow would occur f 2 1 α 4 < 2 149 or α < 2 37 8 10 12, and an overflow would occur f 2 1 α 4 2 128 or α 2 32.25 5 10 9, Although t may be possble to avod an excepton n (14) by dong the dvson before the multplcaton, we cannot apply the same technque to prevent a possble underflow n the calculaton of the numerator n (16): ξ = (µ d 2 1 )/(d2 + µ2 d 2 1 ), where small values of µ and α could cause the product (µ α 2 ) to underflow. As experments, we programmed Algorthms LLL and New n Matlab, whch supports IEEE double precson. In double precson, we would have mnmum exponent value e mn = 1022 and maxmum exponent value e max = 1023. After both programs were run hundreds of tmes wth dentcal random data nput, we observed nether underflows nor overflows and the output results were numercally ndstngushable. ACKNOWLEDGMENTS Ths work s partally supported by Natural Scences and Engneerng Research Councl of Canada. REFERENCES 1. F. T. Luk and D. M. Tracy, An mproved LLL algorthm, Lnear Algebra and Its Applcatons, pp. x x, to appear n 2007. 2. A. Lenstra, H. Lenstra, and L. Lovasz, Factorng polynomals wth ratonal coeffcents, Mathematcsche Annalen 261, pp. 515 534, 1982. 3. B. Hassb and H. Vkalo, On the sphere-decodng algorthm : Expected complexty, IEEE Transactons on Sgnal Processng 53, pp. 2806 2818, 2005. 4. G. Golub and C. V. Loan, Matrx Computatons, 3rd Ed., The Johns Hopkns Unversty Press, Baltmore, MD, 1996.