The Generalized Gauss Reduction Algorithm

Transcription

1 Ž. JOURNAL OF ALGORITHMS 21, ARTICLE NO The Generalized Gauss Reduction Algorithm Michael Kaib and Claus P. Schnorr Fachbereich Mathemati Informati, Uniersitat Franfurt, PSF , Franfurt am Main, Germany Received February 26, 1993 We generalize the Gauss algorithm for the reduction of two-dimensional lattices from the l -norm to arbitrary norms and extend Vallee s analysis 2 J. Algorithms 12 Ž 1991., to the generalized algorithm Academic Press, Inc. Open access under CC BY-NC-ND license. 1. INTRODUCTION Gauss Ga1801 gave, in the language of quadratic forms, an algorithm which reduces a basis a, b of a two-dimensional lattice and finds the two successive minima of the lattice. Vallee Va91 shows that the Gauss 2 2 reduction algorithm performs at most log ŽŽ 2' 2 3.a b. 1' many iterations. This bound is optimal up to an additive constant. Vallee also characterizes for the lattice 2 the minimal size input bases for which the Gauss algorithm performs exactly iterations. The bit complexity of the Gauss algorithm has been studied by Schonhage Sch91 in the language of quadratic forms. While these results are all for the l2-norm, other norms are important, too. The l-norm is the natural norm for integer programming problems. Schnorr Sch93 reduces the problem of factoring integers to a closest lattice vector problem in the l -norm. Lovasz and Scarf LS92 1 propose a generalized basis reduction algorithm that extends the L 3 -algorithm of Lenstra, et al. LLL82 to an arbitrary norm. aib@informati.uni-franfurt.de. schnorr@cs.uni-franfurt.de Copyright 1996 by Academic Press, Inc. Open access under CC BY-NC-ND license.

2 566 KAIB AND SCHNORR Our results. We extend the Gauss reduction algorithm from the l2-norm to an arbitrary norm. This generalized Gauss algorithm Ž gga. essentially coincides with the LovaszScarf algorithm for two-dimensional lattice bases. The gga finds for any norm the two successive minima of the lattice. Given a reduced basis we exhibit minimal size input bases requiring a given number of iterations. These minimal size input bases represent the worst case inputs. They are uniersally worst case for all norms for which the given output basis is reduced. They satisfy the same recursion which holds true for the worst case inputs of the centered Euclidean algorithm according to Dupre Du1846. We show that the generalized Gauss algorithm terminates for any norm after at most log Ž 2' 2 B. ož 1. 1 ' 2 2 many iterations, where B is the maximum of the norms of the two input vectors and 2 is the second successive minimum of the lattice with respect to the given norm. The paper is organized as follows. In Section 2 we introduce reduced lattice bases. In Section 3 we present the generalized Gauss algorithm and its analysis. Section 4 gives complexity bounds for the RAM-model. A preliminary version of this paper has been published by Kaib Ka GEOMETRICAL PRELIMINARIES We generalize the concept of reduced lattice bases for lattices of ran 2 to an arbitrary norm n on. We use the following three elementary lemmata: LEMMA 1. Let a, b n, a 0, let F: n : a b describe a n line in and fž. FŽ.. Then f is a conex function. LEMMA 2. Let F: n be a line in n and 1, 2, 1, 2 be four reals with,,,. Then FŽ. FŽ implies that FŽ. FŽ., and FŽ. FŽ. implies that FŽ FŽ. 2. We will usually apply Lemma 2 in the case 1 0 and 2 1. LEMMA 3. Let M be a closed set in n and 0 M. Then eery point in M with minimal norm lies on the boundary of M. Ž. n n Throughout the paper let a, b be basis of the two-dimensional lattice L a b. We define reduced and well-ordered lattice bases. The reduction algorithm in the next section recurs on well-ordered bases until a reduced basis is found. DEFINITION. A lattice basis Ž a, b. is called reduced if a, b a b a b and well-ordered if a a b b.

3 GENERALIZED GAUSS REDUCTION ALGORITHM 567 By Lemma 2 a b b implies that bab 0. Ž 1. Thus Ž a, b. is well-ordered iff a a b b a b. The ith successie minimum of a lattice L with respect to the norm i is defined as the minimal real such that there are at least i linearly independent lattice vectors of norm at most. Ž. THEOREM 4. If a, b is a reduced basis then a, b are the two successie minima of the lattice L a b. Proof. W.l.o.g. let a b. The theorem claims the following: 4 2 a ra sb for all Ž r, s. Ž 0,0., bra sb for all r, s 0 4. These inequalities follow from the inequalities ab, ara for all r 0 4, bab for all, with, 1. Ž 2. It is therefore sufficient to prove Inequality Ž. 2. For this we show the following. CLAIM. Consider the four dotted areas in Fig. 1. The norm taes its minimum in each of the four dotted areas in the points a b. Inequality Ž. 2 is an immediate consequence of the claim and the reduction conditions b a b. Ž. FIG. 1. Reduced basis a, b.

4 568 KAIB AND SCHNORR Proof of the claim. Each dashed line in the figure contains three lattice points where the middle point has minimal norm; i.e., we have Lemma 2 yields ab a ab, ab b ab. ab ab a, ab ab b for 1. This proves that the points a b have minimal norm for the dotted lines. By Lemma 3 the norm taes its minimum for each of the four dotted areas on the boundary, i.e., on the dotted lines. This proves the claim. 3. ANALYSIS OF THE GENERALIZED GAUSS ALGORITHM We extend the Gauss basis reduction algorithm from the l -norm to an 2 arbitrary norm. We choose the sign of the basis vectors in the algorithm so that the algorithm recurs on well-ordered bases. As a consequence all occurring integral reduction coefficients are positive. THE GENERALIZED GAUSS ALGORITHM Ž gga.. INPUT a well-ordered lattice basis Ž a, b.. WHILE b a b DO 1. b b a, where the integer is chosen to minimize the norm b a. 2. IF a b a b THEN b b. 3. Swap a and b. END WHILE OUTPUT Ž a, b.. Comments. 1. The exchange in Step 3 produces either a well-ordered or a reduced basis. The algorithm traverses, upon exit of Step 3 Žresp. entry of Step 1., a sequence of well-ordered bases until a reduced basis is produced. 2. The algorithm terminates after finitely many steps because the norm of the basis vectors decreases in every, except the last, iteration. 3. To have a well-defined algorithm we require to choose in Step 1 the smallest that minimizes b a.

5 GENERALIZED GAUSS REDUCTION ALGORITHM 569 We associate with an input basis the sequence of lattice bases occurring in the algorithm upon exit of Step 3. The bases of this sequence are all well-ordered, except that the final basis is reduced. If Ž b, c., Ž a, b. are two consecutive bases in any of these sequences we call Ž a, b. the successor basis of Ž b, c. and Ž b, c. a predecessor basis of Ž a, b.. A well-ordered basis has at most one successor basis but may have infinitely many predecessor bases corresponding to runs of the algorithm with various input bases. If Ž b,c. is a predecessor basis of Ž a, b. we call the vector c a predecessor of Ž a, b.. The transition of a well-ordered basis Ž b, c. via Steps 13 to its successor basis Ž a, b. is of the form ž / ž 0/ Ž a, b. Ž b,c. 0 1 ž /ž / Ž b, c. Ž Ž c b., b., Ž 3. where 1 denotes the possible change of sign in Step 2. We see that the predecessor c is of the form c a b. The following lemma characterizes the predecessors of a well-ordered basis Ž a, b.. It generalizes Lemma 1 of Vallee Va91. LEMMA 5. If Ž a, b. is a well-ordered lattice basis then a ector c a b is a predecessor of Ž a, b. if and only if either 1, 2 or 1, 3. Lemma 5 shows that the set of predecessor bases of Ž a, b. does not depend on the norm; i.e., if Ž a, b. is well-ordered for two distinct norms then its two sets of predecessors coincide. Ž. Proof. Since a, b is well-ordered we have aabb. Ž 4. Ž. The predecessor basis b, c is well-ordered iff b b c c.we consider the lines F Ž. a b GŽ. Ž 1. a b H Ž. ba H Ž. ba

6 570 KAIB AND SCHNORR Ž. which are shown in Fig. 2. Inequality 4 implies that FŽ 0. bab FŽ 1. GŽ 0. ab GŽ 1. Ž. Ž. H 0 a ba H 1. Thus by Lemma 2 FŽ. and GŽ. is strictly increasing and H Ž. is increasing Ž i.e., nondecreasing. for 1. This yields a corresponding inequality for H : H Ž 0. GŽ 0. GŽ 1. FŽ 1. FŽ 2. H Ž 1.. We decide for all possible cases of and 1 whether Ž b, c. is well-ordered. 1, 1: Then b c Ž 1. b a H Ž 1. H Ž. abc. Thus Ž b, c. is not well-ordered. 1, 0: Then b c Ž 1. b a H Ž 1. H Ž. abc. Thus Ž b, c. is not well-ordered. Ž. FIG. 2. Well-ordered basis a, b.

7 GENERALIZED GAUSS REDUCTION ALGORITHM 571 0: Then c a b and Ž b, c. is not well-ordered. 1: Then b c a b and Ž b, c. is not well-ordered. 1, 2: Then b c a b b and Ž b, c. is not wellordered. 1, 2: Then c a b H Ž. H Ž 1. a Ž 1. b b c H Ž 1. FŽ. 2 FŽ. 1 b. Thus Ž b,c. is well-ordered. 1, 3: Then cabh Ž.H Ž 1.bcH Ž 2.GŽ. 2 GŽ. 1 b. Thus Ž b, c. is wellordered. For the analysis of the algorithm we consider the sequence of wellordered bases that is traversed upon exit of Step 3. We index this sequence in reverse order Ž b, b.,..., Ž b,b. so that Ž b, b is the input basis, Ž b, b. is the last well-ordered basis, and Ž b, b is the reduced output basis. Let, be the coefficients which, according to Eq. Ž. i i 3, transform Ž b, b. into the successor basis Ž b, b. i i1 i1 i. We have i i 1 Ž b i1, bi. Ž b i, bi1. ž i 0/ 1/ž 2/ ž / Ž b, b1. Ž b 0, b1.. ž The latter matrix product can be expressed by the generalized continuants which Rieger R78 has introduced for the analysis of the centered Euclidean algorithm. These polynomials x2 xn x 2,..., x n, y 1,..., yn y y 1 n are recursively defined as n 0, 1, y, y 1 1 x2 xn x3 xn x4 xn y x. Ž 5 y y 1 y y 2 y y. 1 n n 2 n n1 3 n n2 An easy induction shows that ž / ž /

8 572 KAIB AND SCHNORR Hence 3 2 b1 1 b b. Ž There is a simple formula for the continuants with j 2, j 1: 1 i1 i1 1 1 Ž '. Ž '. Ti Ž i 2 ' 2 Simultaneous induction on i, via Eq. Ž. 5, yields the following inequalities: LEMMA 6. Let j 2 for j 1, and j 3 for j1. Then 2 i 2 i 3 i T i, 2. 1 i i 1 i i 2 i i1 LEMMA 7. Eery sequence of successie well-ordered bases Ž b, b.,..., Ž b,b. satisfies b T b Proof. We see from Lemma 6 that the coefficient of b in Eq. Ž. 1 6 is positive and the coefficient of b0 has sign 1. We distinguish two cases: Case We have 2 b b T b The first inequality follows from Eq. Ž. 6 by Inequality Ž. 1 since Ž b, b. 0 1 is well-ordered. The second inequality holds by Lemma 6. Case We have b b since Ž b, b is well-ordered. Therefore Eq. Ž. 6 and the triangular inequality yields ž 1 2 / 2 3 b b 1 1. The right-hand factor can be simplified to Ž To verify this equation develop its first and last term via Eq. Ž. 5. Finally the claim follows from Lemma 6 since 1 3. We consider the number of iterations of the gga or equivalently the number of traversed well-ordered bases. We bound this number as a function of B b where b is the largest input vector. 2

9 GENERALIZED GAUSS REDUCTION ALGORITHM 573 THEOREM 8. The gga performs on input Ž a, b., for B b2, at most log Ž 2' 2 B. ož 1. 1 ' 2 many iterations where 2 is the second successie minimum of the lattice. Remar. If the input basis is not well-ordered there may be an extra iteration. The o-term is at most 2 log Ž 4' 2. 1 ' where the maximum occurs in case of a single iteration. Proof. Let Ž b, b. be a well-ordered input basis and Ž b, b the output basis. There are 1 iterations. Lemma 7 tells us that Ž. Equation 7 implies that and thus we have for B. b 1 T B. b ' 2 2 Ž '. Ž ' B, Ž '. 1 log1' B ož 1. The bound of Theorem 8 is optimal for all norms: THEOREM 9. Let Ž b, b. be a reduced basis with b b and bi1 bi1 2bi for i 0,...,. Then the gga performs on input Ž b, b. exactly 1 iterations where 1 log ' Ž 2 B. 1 ož and B b. Remars. 1 2 The difference of the upper and lower bound in the above two theorems is 1 log Ž ' 2. ož ož. 1' 2 1. In the particular case that b 1, b0 are the integers b1 0, b0 1 the recursion bi1 bi1 2bi for i 0,..., yields, according to Dupré Du1846, the minimal integers b, b1for which the centered Euclidean algorithm performs exactly 1 divisions. Vallee Va91 has extended this minimality, in the case of the l2-norm, to bases of the lattice 2. The novelty in Theorem 9 is that the recursion bi1 bi1 2bi for 0 1,..., is valid for all norms, all lattices, and all reduced output bases Ž b, b For the proof we characterize the well-ordered predecessor bases Ž b, c., cab, of reduced bases Ž a, b.. This extends Lemma 5.

10 574 KAIB AND SCHNORR Ž. LEMMA 10. Let a, b be a reduced basis and c a b where 1 and. 1. If a b and Ž,. Ž 1, 2. then Ž b, c. is well-ordered iff If b a then Ž b, c. is not well-ordered for 0 and wellordered for 1, Ž , and for 1, 22 1, where Ž ab.. i i Proof of Theorem 9. The bases Ž b, b. i i1 for i 0,..., are all well- ordered. This holds by Lemma 10 for i 0 and by Lemma 5 for i 0. Hence the gga performs on input b, b1 exactly 1 iterations with all reduction coefficients equal to 2, and then finds the reduced basis Ž b,b.. Equation Ž 6. implies that 1 0 b T b Tb Ž. Thus we see from b, b 3 and Eq. 7 that b Ž T 3T Ž ' 2 '. Ž. 2Ž Ž. ' o 1, 2 2 hence log ' Ž 2 B. 2 ož Proof of Lemma 10. We collect facts that cover all the claims. The basis Ž b,c. is well-ordered iff b a Ž 1. b ab. Ž 8. If a b the left-hand inequality holds by Theorem 4 iff 1. This inequality is trivial for b a. For the right-hand inequality we consider the line HŽ. a b, assuming that Ž a, b. is reduced. We have HŽ 1. a b a H0 Ž.ab H1 Ž.. For 0 Lemma 2 implies that a Ž 1. b HŽ 1. HŽ. ab. Ž. Hence b, c is not well-ordered. This proves all claims in the case 0. Now let 1. We have a Ž 1. b HŽ 1. HŽ. ab. Ž 9.

11 GENERALIZED GAUSS REDUCTION ALGORITHM 575 If 1 we have a c which shows that b ac c. Ž 10. If 1 the inequality b a a b HŽ. 1 c implies that 1 2 b Ž ba. c c. Ž Now assume that the left-hand Inequality Ž. 8 holds but Ž b, c. is not well-ordered. In this case equality must hold in Inequality Ž. 9. This implies that Ž b, c. is reduced and thus b, c. Moreover if a b 1 2, i.e., the right-hand Inequality Ž. 8 does not hold, we have b c 2, and thus the inequalities above imply that 1 for 1 and 2 for 1. On the other hand, if b a, Inequality Ž 10. yields 2Ž. and Inequality Ž 11. yields 2Ž The claims in case 1. For those, where it is claimed that Ž b, c. is well-ordered we have shown, assuming that Ž b, c. is not well-ordered, an inequality excluding this. Moreover, in the case that a b, we have shown that Ž b, c. is not well-ordered if 1. Remar. In the indefinite cases of Lemma 10, i.e., ab, 1, 2, ba, 1, 1 2Ž. 2 1, ba, 1, 1 2Ž , the above proof shows that Ž b, c. is either reduced or well-ordered. Both reduced and well-ordered bases do actually occur, as the norm and the lattice vary, in each of the three cases. However there is only one indefinite case for the Euclidean norm, namely b a and 1, TIME BOUNDS The generalized Gauss algorithm described in the last section needs n access to a norm oracle which for given a outputs a. We give time bounds for the RAM model with the arithmetic operations multiplication, division, addition, subtraction, comparison, and next integer computation at unit costs. We count for steps arithmetic steps and oracle calls. In this section we prove the following.

12 576 KAIB AND SCHNORR THEOREM 11. Gien an oracle for an arbitrary norm there is an n algorithm which -reduces a gien basis a, b using OŽn logžn. log B. many steps where B maxža, b n Efficient -reduction. For an efficient reduction of a basis a, b in an arbitrary norm we first reduce a, b in the norm corresponding to a suitable inner product ²:, and we subsequently reduce the resulting basis in the -norm. We initially perform a ²:, -reduction since it only costs O1 Ž. arithmetic steps per iteration. The inner product ²:, is chosen so that x n x14is spherical in the sense that max nx² y, y: 12 Žy² x, x: 12. On Ž 1.5. x, y. The exis- tence of ²:, follows from Le83, see construction of in Section 2, pp. 542 ff. We assume that the inner product is given, we do not count the steps for producing it. The constant B maxž² a, a : 12, ² b, b: 12. ²,: 2, ²,: satisfies B On Ž 1.5 B.. ²,: ²: Ž. The initial,-reduction in O n log B arithmetic steps. In each iteration we transform the Gram matrix Gž ² a, a: ² a, b: / ² b, a: ² b, b: and the transformation matrix H GL Ž. satisfying Ž a, b. n current current Ž. a, b H as G S GS, HHS where input input 1 S ž 0/ and is the integer closest to ² a, b: ² a, a :. Each iteration requires six multiplications, six subtractions, one division, and one next integer computation. The initial Ž resp. final. transformations of Ž a, b. into G Žresp. bac from H into Ž a, b.. require 7n multiplications and 5n 3 additions. According to Theorem 8 the entire ²:, -reduction of a, b is done in OŽ log B. OŽlogŽ B n.. iterations. ²,: Computing in OŽ 1. resp. OŽ log. 2 1 oracle steps. Let n n : denote the function that minimizes b Ž a, ba.. For a given well-ordered basis a, b we have to compute b xa where Ž x, a. is the successor basis of Ž a, b.. By the inequality baxa we can compute, via the bisection method, using OŽlogxa. oracle steps. Except for the final iteration we always have xa and the step bound is OŽ. 1. For the final iteration we have xa. 2 1

13 GENERALIZED GAUSS REDUCTION ALGORITHM 577 The final -reduction in OŽn logž n steps. It follows from Theorem 8 that the final -reduction requires at most ž / x² y, y: ' 12 log 2 2 max n 1' 2 x, y 1 ož 1. OŽ log n 12. y ² x, x: many iterations. Every iteration, except the final one, requires OŽ. 1 norm computations and On Ž. arithmetic steps. The final iteration costs OŽ log. norm computations and Onlog Ž. arithmetic steps The case of the l Ž l. 1 -norm. There are particularly efficient algorithms to compute for the l1- and l-norm. For the l1-norm the real t minimizing b ta1 is the generalized median, with weights a i, of the component fractions biai for i 1,...,n which can be computed using On Ž. arithmetic steps. For the l -norm the graph of the function b ta with real indeterminate t is the maximum polygon of the 2n lines Žb i ta.. We sort, using Onlog Ž n. i arithmetic steps, these lines in order of descending gradient. A subsequent scan of the lines computes, using On Ž. arithmetic steps, the vertices of the polygon and in particular its minimal point which yields the real t that minimizes b ta. Details can be found in KS93. Hence the l Ž l.-norm reduction of a, b taes at most OŽ log B n log n. 1 arithmetic steps where B maxža, b. and is the l Ž l.-norm. 2 1 ACKNOWLEDGMENT We are grateful to Joos Heintz for encouragement and critical comments. REFERENCES Da93 Du1846 Ga1801 Ka91 KS93 Le83 H. Daude, Des fractions continues a la reduction des reseaux: Analyse en moyenne, These ` de doctorat, Universite de Caen, A. Dupre, Sur le nombre de divisions `a effectuer pour obtenir le plus grand commun diviseur entre deux nombres entiers, J. Math. 11 Ž 1846., C. F. Gauss, Disquisitiones Arithmeticae, Leipzig German translation: Untersuchungen uber die hohere Arithmeti, Springer, Berlin ŽReprint: Chelsea, New Yor, M. Kaib, The Gauß lattice basis reduction algorithm succeeds with any norm, in Proceedings of the FCT 91, Springer Lecture Notes in Computer Science, Vol. 529, pp , Springer-Verlag, Berlin, M. Kaib and C. P. Schnorr, The generalized Gauss reduction algorithm, Technical Report, Universitat Franfurt, H. W. Lenstra, Jr., Integer programming with a fixed number of variables. Math. Oper. Res. 8Ž.Ž 4, 1983.,

14 578 KAIB AND SCHNORR LLL82 A. K. Lenstra, H. W. Lenstra, Jr., and L. Lovasz, Factoring polynomials with rational coefficients, Math. Ann. 261 Ž 1982., LS92 L. Lovasz and H. Scarf, The generalized basis reduction algorithm, Math. Oper. Res. 17Ž.Ž , R78 G. J. Rieger, Uber die mittlere Schrittzahl bei Divisionsalgorithmen, Math. Nachr. 82 Ž 1978., Sch93 C. P. Schnorr, Factoring integers and computing discrete logarithms via diophantine approximation, in Advances in Computational Complexity ŽJim-Yi Cai, Ed.., DIMACS Series in Discrete Mathematics and Theoretical Computer Science, pp , Amer. Math. Soc., Providence, RI, Sch91 A. Schonhage, Fast reduction and composition of binary quadratic forms, in Proc. ISSAC 1991 Ž S. M. Watt, Ed.., pp , Assoc. Comput. Mach., New Yor, Va91 B. Vallee, Gauss algorithm revisited, J. Algorithms 12 Ž 1991., VF90 B. Vallee and Ph. Flajolet, The lattice reduction algorithm of Gauss: An average case analysis, in Proc. 31st IEEE Symposium on Foundations of Computer Science, 1990, pp