Fast algorithms for polynomials and matrices Part 2: polynomial multiplication

Transcription

1 Fast algorithms for polynomials and matrices Part 2: polynomial multiplication by Grégoire Lecerf Computer Science Laboratory & CNRS École polytechnique Palaiseau Cedex France 1 Notation In this part R is a commutative ring with unity. Let f f 0 + f 1 x + + f n x n,andg g 0 + g 1 x + + g n x n. Let h = h 0 + h 1 x + + h 2n x 2n fg. 2

2 Faster product of two polynomials of degree 1 We want to multiply f = f 0 + f 1 x and g = g 0 + g 1 x. Remark 1. The naive product requires 4 products in R. fˆ ĝ ĥ (f(0),f(1),f( )) (g(0),g(1),g( )) ( fˆ1 ĝ 1,fˆ2 ĝ 2,fˆ3 ĝ 3 ). 1 0 ) 1 1 ( f0 f, ) 1 1 ( g0 g Lemma 2. ĥ =(h(0),h(1),h( )) h0 h 1 h 2, whereh = fg. Proof. h(a)=f(a) g(a) for all a {0, 1, }. Remark 3. The product can thus be done with only 3 multiplications in R. 3 Karatsuba algorithm For any polynomial f, wewritef i j 0 f 2j+i x j,thatis: F 0 f 0 + f 2 x + f 4 x 2 +, F 1 f 1 + f 3 x + f 5 x 2 +, We also introduce F (x, y) F 0 (y) +F 1 (y) x, sothatf(x) =F (x, x 2 ) and can apply the latter trick to multiply F and G of degree 1 when seen in (R[y])[x]. Algorithm 1 Input. f and g in R[x] of degree at most n. Output. h fg. 1. If n =0 then return f 0 g Otherwise, let m n/2 (that is the integer part of n/2), and compute: F 0 (y) f 0 + f 2 y + f 4 y f 2m y m and F 1 f 1 + f 3 y + f 5 y f 2m+1 y m, G 0 (y) g 0 + g 2 y + g 4 y g 2m y m and G 1 g 1 + g 3 y + g 5 y g 2m+1 y m. 3. Compute Fˆ(x, y) =(F 0 (y), F 0 (y) +F 1 (y), F 1 (y)), andĝ(x, y) =(G 0 (y), G 0 (y) +G 1 (y), G 1 (y)). 4. Recursively compute Ĥ = ( Fˆ0(y) Ĝ 0 (y),fˆ1(y) Ĝ 1 (y),fˆ2(y) Ĝ 2 (y) ). 5. Deduce H 0 = Ĥ 0, H 1 = Ĥ 1 Ĥ 0 Ĥ 2, H 2 = Ĥ Return h(x) H 0 (x 2 )+xh 1 (x 2 )+x 2 H 2 (x 2 ). 4

3 Analysis of Karatsuba algorithm Proposition 4. [ Karatsuba, Ofman, 1963]The Karatsuba algorithm is correct and takes O(n log2 3 ) O(n 1.59 ) operations in R. Proof. The correctness is proved easily by strong induction on n. Let K(n) represent the cost in size n. There exists a constant γ such that we have that K(n) 3 K( n/2 )+γ n. We are searching for two positive constants α and β such that K(n) α n log2 3 β n and 3 α n/2 log2 3 3 β n/2 + γ n α n log2 3 β n, hold for n sufficiently large. From n/2 n/2 + 1/2, byconvexitywehavethat (n/2 + 1/2) log2 3 (n/2) log2 3 log2 (3) 1 log 2 (3) (n/2 + 1/2) which can be bounded by ε n, ifn is larger than a suitable value N ε.thereforeitisnowsufficient to get 3 εαn 3 β n/2 + γ n β n, whichrewritesinto0 (β/2 γ 3 εα) n. We can thus take β =2(3εα+ γ), sothatitremainstofixα in order to K(n) α n log2 3 2(3εα+ γ) n holds for n {1,,N ε }. For instance we set ε 1/12. Letn be a fixed value in {1,,N ε }.Sinceαn log2 3 2(3εα+ γ) n tends to infinity when α goes to infinity, and since the derivative of the latter expression in α is n log2 3 1/2 1/2, we can chose a suitable value of α to conclude the proof. 5 Product of two polynomials of degree 2 Suppose we want to multiply f = f 0 + f 1 x + f 2 x 2 and g = g 0 + g 1 x + g 2 x 2. Remark 5. The naive product requires 9 products in R. fˆ ĝ ĥ (f(0),f(1),f( 1),f(2),f( )) (g(0),g(1),g( 1),g(2),g( )) ( fˆ0 ĝ 0,,fˆ4 ĝ 4 ). Lemma 6. ĥ =(h(0),h(1),h( 1),h(2),h( )) f 0 f f g 0 g g , h h h h h 4,whereh = fg. Proof. h(a)=f(a) g(a) for all a {0, 1, 1, 2, }. Remark 7. If 2 is invertible then the latter matrix is invertible. Remark 8. The product can thus be done with only 5 multiplications in R plus several scalar operations. 6

4 The Toom Cook algorithm For any polynomial f, wewritef i j 0 f 3j+i x j,thatis: F 0 f 0 + f 3 x + f 6 x 2 +, F 1 f 1 + f 4 x + f 7 x 2 +, F 2 f 2 + f 5 x + f 8 x 2 +. We also introduce F (x, y) F 0 (y)+f 1 (y) x + F 2 (y) x 2. We have that f(x)=f (x, x 3 ) and can apply the latter trick on F and G seen as polynomials of degree 2 in (R[y])[x]. Algorithm 2 Input. f and g in R[x] of degree at most n. Output. h fg. 1. If n =0 then return f 0 g Otherwise, let m n/3, andcomputef 0, F 1, F 2, G 0, G 1, G Compute Fˆ and Ĝ. 4. Recursively compute Ĥ ( ) Fˆ0 Ĝ 0,,Fˆ4 Ĝ Deduce H from Ĥ and return H(x,x 3 ). 7 Analysis of the Toom Cook algorithm Proposition 9. [ Toom 1963, Cook 1966] If 2 is invertible in R (with given inverse), then the polynomial product in degree n can be done with O(n log3 5 ) O(n 1.47 ) operations in R. Proof. Left as an exercise. Proposition 10. For any ε > 0, ifk is a field with sufficiently many elements then the product in degree n can be done with O(n 1+ε ) operations in K thebig-ohactuallydependsonε. 8

5 Fast Fourier transform From now on we assume that R is a K-algebra, that n =2 k,andthatk has a primitive root of unity ω of order 2 k. The discrete Fourier transform (with respect to ω) off R[x] of degree at most n 1 is: DFT ω,n : f R[x] n 1 (f(1),f(ω),f(ω 2 ),,f(ω n 1 )) =: fˆ. Algorithm 3 Input. f of degree at most n 1, andω of order n. Output. fˆ 1. If n =1 then return (f 0 ). 2. Let m n/2 and compute g(x) m (f i + f i+m ) x i, h(x) i=0 m (f i f i+m ) ω i x i. i=0 3. Recursively compute ĝ DFT ω 2,m(g) and ĥ DFT ω 2,m(h). 4. Return ( ĝ 0,ĥ 0,ĝ 1,ĥ 1,,ĝ m 1,ĥ m 1 ). 9 Analysis of the fast Fourier transform Proposition 11. Algorithm 3 is correct and takes O(n log n) additions in R and O(n log n) scalar multiplications by elements in K. Proof. If l =2j is even then f(ω l )= m i=0 (f i + ω lm f i+m ) ω li = g(ω l )=ĝ j. If l =2j +1 is odd then f(ω l )= m (f i=0 i + ω lm f i+m ) ω li = h(ω 2j )=ĥ j. Let F (n) be the cost function. There exists a positive constant γ 1 such that: F (n) 2 F (n/2) + γ n. We are looking for positive constants α and β such that F (n) α n log 2 n + β n. Itissufficient to obtain α n log 2 (n/2) + γ n α n log 2 n, which is equivalent to α n log 2 (1/2) + γ n 0. We can thus take α = γ, andsetβ to 1 so that F (n) γ n log 2 n + β n holds for n =1. 10

6 Inverse transform Proposition 12. n DFT 1 ω,n (v)=dft ω 1,n(v). Proof. Left as an exercise. Proposition 13. If n =2 k,if 2 is invertible, and if a primitive root ω of order n is given, then the product of two polynomials whose degree sum is at most n 1 can be computed with O(n log n) operations and scalar operations in R. Proof. nfg= DFT ω 1,n(DFT ω,n (f) DFT ω,n (g)) (coordinatewise product). 11 FFT over suitable finite fields Proposition 14. The finite field F q with q elements contains a n-th root of unity if, and only if, n divides q 1. Example 15. The number p is prime. This allows to multiply polynomials in F p [x] up to output degree , whichissufficient in practice according to the usual available memory size of computers. Remark 16. Primitive roots can be chosen at random with a reasonable probability of success. 12

7 Special Chinese remaindering for Z/m Z Let p 1, p 2 and p 3 be three odd prime numbers m, suchthat2 k divides p 1 1, p 2 1, andp 3 1. Let P = p 1 p 2 p 3. Given r 1 {0,,p 1 1}, r 2 {0,,p 2 1}, andr 3 {0,,p 3 1}. There exists a unique integer r {0,,P 1} such that r r 1 mod p 1, r r 2 mod p 2,andr r 3 mod p Three primes FFT for Z/m Z Let f and g be two polynomials in Z/m Z[x] such that the degree of fg is at most 2 k 1. Let F and G be the preimages of f and g in Z[x]. 1. Compute H 1 as the preimage of FG modulo p Compute H 2 as the preimage of FG modulo p Compute H 3 as the preimage of FG modulo p Compute the unique polynomial H with coefficients in {0,,P 1} that equals H i mod p i for all i {1, 2, 3}. Proposition 17. If 2 k (m 1) 2 <P then fg= H mod m. Proof. The coefficients of FH are nonnegative integers that are at most 2 k (m 1) 2. Example 18. With p , p ,andp ,onecanmultiplypolynomials over Z/m Z for all m<2 64 and up to outdegree

8 Introducing virtual roots of unity In order to benefit of the FFT over any ring, Schönhage and Strassen, proposedtointroducevirtual roots of unity as follows: Lemma 19. If 2 is invertible in R and if n is a power of 2 then the class of x in A = R[x]/(x n +1) is a 2 nth primitive root of unity. Proof. It is clear that x is a 2 nth root of unity in R. Ifx were not primitive then x t 1 would be zero for astrictdivisort of 2 n. Thereforet divides n, whencex n =1holds in R, whichisnotpossiblesince2 is assumed to be a unit. 15 Fast universal polynomial product Let R be a ring with unity, where 2 is invertible and whose inverse is given. Algorithm 4 Input. f and g of degrees at most n 1, wheren =2 k. Output. fg. 1. Set d 2 k/2. 2. Rewrite f and g into F (x, y) F 0 (x)+f 1 (x) y + + F d (x) y d 1, G(x, y) G 0 (x)+g 1 (x) y + + G d (x) y d 1, so that the F i and G i have degree at most d 1, and f(x)=f (x, x d ), g(x)=g(x, x d ). 3. Compute H FG in A[y], wherea R[x]/(x 2d +1) via FFT and using the present algorithm recursively to compute products in A. 4. Return H(x, x d ). 16

9 Analysis of the fast universal polynomial product Proposition 20. Algorithm 4 works correctly as specified, and takes O(n log n log log n) operations in R. Proof. (Sketch) Since the class of x in A has order 4 d. TheFFTiny can be used to compute FG seen in A[y]. ThelatterproductcanbeliftedinR[x][y] since the degree in x of F and G is at most 2(d 1). Steps 2 and 4 take O(n) operations. Step 3 performs: O(d log d) additions and subtractions in A, whichreducetoo(d) operations in R, O(d log d) multiplication by a power of ω, whichreducestoo(d) operations in R, O(d) products in A. Let F (n) represent the cost function. It satisfies the recurrence F (n) df(d)+αnlog n for some positive constant α. Letting Φ(k) =F (2 k )/2 k,thelatterinequalityrewritesintoφ(k) Φ( k/2 ) +α k,fork sufficiently large, which concludes the proof. 17 Extension to any characteristic Remark 21. If 2 is not invertible but 3 is invertible then we can do similar computing via a triadic adaptation of the FFT. Remark 22. If neither 2 and 3 are known to be invertible then we compute 2 k times the product by performing the later algorithms without dividing by 2. Wealsocompute3 l times the product. From a Bézout relation u 2 k + v 3 l =1 we deduce the product. Remark 23. The cost of the FFT is essentially constant between two consecutive powers of n. Thiscost can be smoothen thanks to the truncated Fourier tranform, proposed by van der Hoeven. 18

10 The multiplication cost function We denote by M(n) a cost function for multiplying two univariate polynomials of degree n over an arbitrary commutative ring R with unity, in terms of the number of arithmetic operations in R. Naive Karatsuba Schönhage & Strassen & Cantor & Kaltofen O(n 2 ) O(n log23 ) O(n log n log log n) We assume that M(n)/n is increasing and that M(mn) m 2 M(n) holds for all positive integers m and n. This is satisfied by the above cost functions. Exercise 1. Let f R[x] and let a R. Showthatf(a + x) can be computed with O(M(n) log n) operations in R. 19 Multi-product Algorithm 5 Input. Polynomials f 1,,f r in R[x]. Output. f 1 f r. 1. If r =1 then return f Let h = r/2 and compute recursively g 1 f 1 f h and g 2 f h+1 f r. 3. Return g 1 g 2. Proposition 24. Algorithm 5 is correct and takes O(M(n) log 2 r) operations in R, wheren r i=1 deg f i. Proof. f 1 f r f 1 f h f h+1 f r f 1 f h f h +1 f h f h+1 f h f h +1 f r The depth of this tree is in O(log r). The sum of the degree of a given depth is always at most r i=1 f i. 20

11 Similarities with integers (1/3) Definition 25. Let B be a power of 2. Any integer a N can be uniquely represented by its expansion a = i 0 a i B i in base B, witha i {0,,B 1}. Thedense B-adic representation of an integer a<b n is the vector (a 0,,a n ). Remark 26. It is usual to take B =2in theory, but in practice it is better to take large values of B around Proposition 27. a<b n and b<b n then their product can be computed with O(n 2 log 2 B) bit-operations. Exercise 2. Adapt the Karatsuba algorithm for multiplying integers. 21 Similarities with integers (2/3) Suppose we want to multiply two integers a<b n and b<b n given in dense B-adic representation. Let f(x) i 0 a i x i and g(x) i 0 b i x i. Using the Schönhage and Strassen algorithm for computing h in Z/m Z with m n (B 1) fg takes O(n log n loglog n) operations In fact this approach can be improved by adapting the Schönhage and Strassen algorithm directly to N, thusleadingto: Proposition 28. [ Schönhage and Strassen, 1971]The product of two integers of n bits can be done with O(n log n log log n) bit-operations. Proposition 29. [ Fürer, 2007]The product of two integers of n bits can be done with O(n log n log n) bit-operations. Here log represents the iterated logarithm of n. 22

12 Similarities with integers (3/3) Similarly to polynomials, we introduce the const function I(n) for multiplying two integers of bit-size at most n in binary representation. Naive Karatsuba Schönhage & Strassen Fürer O(n 2 ) O(n log23 ) O(n log n log log n) O(n log n 2 log n ) We assume that I(n)/n is increasing and that I(mn) m 2 I(n) holds for all positive integers m and n. This is satisfied by the above cost functions. 23 Dense representation of Multivariate polynomials Any polynomial f in R[z 1,,z n ] is made of a sum of terms, with each term composed of a coefficient and an exponent seen as a vector in N n.foranexponente =(e 1,,e n ) N n e,themonomialz 1 e 1 z n n will be written z e.foranye N n,weletf e denote the coefficient of z e in f. Definition 30. The support of f is defined by supp f = {e N n f e 0}. A block is a subset of N n of the form n j=1 {0, 1,,d j 1}, withd 1,,d n N. Givenapolynomial f R[z 1,,z n ],itsblock support is the smallest block such that: n dsupp(f)= j=1 {0, 1,,d f,j } with supp(f) dsupp(f). Inotherwords,assumingd f 0,wehaved f,j = deg zj f +1 for j =1,,n.We will denote by d f = d f,1 d f,n the cardinality of dsupp(f). Definition 31. The dense block representation of f,isthedataofthed f,j and of the vector of the coefficients corresponding to the monomials of dsupp(f) in lexicographic order. 24

13 The Kronecker substitution For computing h = fg,thekroneckersubstitutionisdefinedasfollows: K dh : R[z 1,,z n ] R[x] f f(x,x dh,1,x dh,1dh,2,,x dh,1 dh,n 1 ). We compute K dh (f) and K dh (g), performtheirproduct,andrecoverh by h = K 1 dh (K dh (f) K dh (g)). Proposition 32. Assuming the block dense representation, the product h = fg can be computed using M(d h ) operations in R. Exercise 3. Multiply f x + y by g x y +1 by using the Kronecker substitution. 25 The Kronecker substitution over the integers (1/2) Over the integers, namely when R = Z, onecanfurtherapplythekroneckersubstitutiontoreduceto the multiplication of two large integers. For any integer a we write l a = log 2 ( a +1) for its bit-size, and denote by l f = max e l fe the maximal bit-length of the coefficients of f (and similarly for g and h). Since max e we have l h η l f + l g + l min(df,d g ). h e min (d f,d g ) max e f e max g e, e The coefficients of h thus have bit-length at most η. Wewillbeabletorecoverthem(withtheirsigns)from an approximation modulo 2 η+1.thesubstitutionworksasfollows: K dh,η: Z[z 1,,z n ] Z f K dh (f)(2 η+1 ). One thus computes K dh,η(f) and K dh,η(g), doestheintegerproduct,andrecovers h = K 1 dh,η(k dh,η(f) K dh,η(g)). Exercise 4. Multiply f x + y by g x y +1 by using tha latter Kronecker substitution. 26

14 The Kronecker substitution over the integers (2/2) Corollary 33. With the above block dense representation, the product h of f times g in Z[z 1, O(I(η d h )+ni(log d h )+(d f +d g ) log d h ) bit-operations.,z n ] takes Proof. The evaluation at 2 η+1 takes linear time thanks to the binary representation of the integers being used. Remark 34. In a similar way, we may use the Kronecker substitution for the multiplication of polynomials with modular coefficients in R = Z/p Z, p {2, 3, }. Indeed,wefirstmapf,g R[z 1, z n ] to polynomials in {0,,p 1}[z 1,,z n ] Z[z 1,,z n ],multiplythemasintegerpolynomials,andfinallyreducemodulo p. 27