Fast reversion of power series

Size: px

Start display at page:

Download "Fast reversion of power series"

Belinda Brooks
5 years ago
Views:

1 Fast reversion of power series Fredrik Johansson November 2011

2 Overview Fast power series arithmetic Fast composition and reversion (Brent and Kung, 1978) A new algorithm for reversion Implementation results

3 Operations on power series Let R be a commutative ring (with 1). We consider arithmetic in the ring of truncated formal power series P = R[[x]]/ x n. n 1 a = a(x) = a k x k, a k = [x k ]a(x) R. k=0 Addition h(x) = f(x)+g(x) = n 1 k=0 (f k +g k )x k Multiplication h(x) = f(x)g(x) = ( n 1 k ) k=0 i=0 f ig k i x k Reciprocal If f 0 is a unit, h(x) such that f(x)h(x) = 1 Composition Reversion If g 0 = 0, h(x) = f(g(x)) = n 1 k=0 f k(g(x)) k If f 0 = 0 and f 1 is a unit, h(x) s.t. f(h(x)) = x

4 Computational complexity We measure complexity (as a function of n) for an operation in P by counting the number of ring operations in R. M(n): multiplying two length-n polynomials Classical M(n) = O(n 2 ) Karatsuba M(n) = O(n log 2 (3) ) = O(n 1.59 ) Fast Fourier Transform M(n) = O(nlog 1+o(1) n) = O(n 1+o(1) n) MM(n) = O(n ω ): multiplying two n n matrices Classical MM(n) = O(n 3 ) Strassen MM(n) = O(n log 2 (7) ) = O(n 2.81 ) Coppersmith-Winograd MM(n) = O(n 2.38 )

5 Newton iteration Let g k P and ϕ P[t] be such that ϕ (g k ) is invertible and ϕ(g k ) 0 mod x m. If g k+1 P is a solution of g k+1 g k ϕ(g k) ϕ (g k ) mod x2m, then ϕ(g k+1 ) 0 mod x 2m, g k+1 g k mod x m, and ϕ (g k+1 ) is invertible. O(M(n)) computation of 1/f: take ϕ(g) = 1 g iteration g k+1 = 2g k fgk 2. f, giving the

6 Reversion using Newton iteration Let f P with [x 0 ]f = 0 and [x 1 ]f invertible in R. Define ϕ by ϕ(g(x)) = f(g(x)) x. Then ϕ(f 1 (x)) = 0, so the compositional inverse (reversion) f 1 is a root of ϕ. Quadratically convergent iteration: g k+1 = g k f(g k) x f. (g k ) Up to constant factors, composition reversion (Brent and Kung, 1978).

7 Fast composition Can we reduce composition to multiplication?

8 Composition by elementary functions Assume that 1,2,...,n 1 are invertible in R. n 2 f (x) = (k +1)f k+1 x k k=0 log(1+f(x)) = f (x) 1+f(x) n 1 ( ) 1 f(x) = f k 1 x k k k=1 atan(f(x)) = f (x) 1+(f(x)) 2 Newton iteration: log exp, atan tan. sin, cos, sinh, cosh, asin... using algebraic transformations. Cost: O(M(n)) = O(n 1+o(1) ). Generalization: differential equations (hypergeometric,...)

9 Composition of general power series Horner s rule f(g(x)) = f 0 +g(f 1 +g(f 2 + +g(f n 2 +f n 1 x) )) Complexity: O(nM(n)) Brent-Kung 2.1: baby-step, giant-step version of Horner s rule Complexity: O(n 1/2 M(n)+n 1/2 MM(n 1/2 )) Brent-Kung 2.2: divide-and-conquer Taylor expansion Complexity: O((nlogn) 1/2 M(n))

10 Brent-Kung algorithm 2.1 Example with n = 9, m = n = 3 Computing f(g(x)) where f(x) = f 0 +f 1 x +...+f 8 x 8 (f 0 +f 1 g +f 2 g 2 )+(f 3 +f 4 g +f 5 g 2 )g 3 +(f 6 +f 7 g +f 8 g 2 )g 6 (m m) (m m 2 ) matrix multiplication: f 0 f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8 1 g g 2 Final combination using block Horner s rule.

11 Complexity of composition algorithms Horner s rule: O(nM(n)) BK 2.1: O(n 1/2 M(n)+n 1/2 MM(n 1/2 )) BK 2.2: O((nlogn) 1/2 M(n)) = O(n 3/2+o(1) ) M(n) MM(n) Horner BK 2.1 BK 2.2 n 2 n 3 n 3 n 2.5 n 2.5 log 0.5 n n 1.59 n 3 n 2.59 n 2.09 n 2.09 log 0.5 n nlogn n 3 n 2 logn n 2 n 1.5 log 1.5 n nlogn n 2.38 n 2 logn n 1.69 n 1.5 log 1.5 n (nlogn) (n 2 ) (n 2 logn) (n 1.5 logn) (n 1.5 log 1.5 n) BK 2.1 wins for small n, BK 2.2 wins for large n BK 2.1 wins with hypothetical O(n 2 ) matrix multiplication

12 The Lagrange inversion theorem If f(x) = x/h(x) then the compositional inverse or reversion f 1 (x) satisfying f(f 1 (x)) = f 1 (f(x)) = x exists and its coefficients are given by [x k ]f 1 (x) = 1 k [xk 1 ]h(x) k. O(nM(n)) algorithm: compute x/h(x) using Newton iteration for the reciprocal, then compute h,h 2,h 3,... using successive multiplications Better than classical algorithms (O(n 3 )) Equivalent to Newton iteration + Horner s rule Worse than Newton iteration + BK2.1 or BK2.2

13 Fast Lagrange inversion For 0 < k < n, write k = im+j. Using the definition of the Cauchy product, we can extract a single coefficient in O(n) operations: [x k ]h k = k r=0 ( ) ([x r ])h im [x k r ] h j We only need to compute h j where j = 1,2,...,m 1 and h im where i = 1,2,...,n/m. Choose m n only O( n) polynomial multiplications. We replace O(n) full polynomial multiplications with O(n 2 ) coefficient operations.

14 Fast Lagrange inversion, first version 1: m n 2: h x/f mod x n 1 3: for 1 i < m do 4: h i+1 h i h mod x n 1 5: b i 1 i [xi 1 ]h i 6: end for 7: t h m 8: for i = m,2m,3m,...,lm < n do 9: b i 1 i [xi 1 ]t 10: for 1 j < m while i +j < n do 11: b i+j 1 i+j i+j 1 k=0 ([xk ]t) ([x i+j k 1 ]h j ) 12: end for 13: t t h m mod x n 1 14: end for 15: return b 1 +b 2 x +...+b n 1 x n 1

15 Using matrix multiplication Take n = 8, m = 8 1 = 3. We need the coefficients b 1,b 2,...,b 7 of 1,x,...,x 6 in powers of h. If hi k [x i ]h k : h2 0 h1 0 h A = h5 3 h4 3 h3 3 h2 3 h1 3 h (h8 6) (h6 7 ) h6 6 h5 6 h4 6 h3 6 h2 6 h1 6 h h0 1 h1 1 h2 1 h3 1 h4 1 h5 1 h 1 6 B = 0 h0 2 h1 2 h2 2 h3 2 h4 2 h5 2 h6 2 (h7 2) h0 3 h1 3 h2 3 h3 3 h4 3 h5 3 h6 3 (h7 3) (h3 8 ) b 1 b 2 b 3 AB T = b 4 b 5 b 6 b 7 Example: b 5 = h 3 4 h2 0 +h3 3 h2 1 +h3 2 h2 2 +h3 1 h2 3 +h3 0 h2 4

16 Fast Lagrange inversion, matrix version 1: m n 1 2: h x/f mod x n 1 3: {Assemble m m 2 matrices B and A from h,h 2,...,h m and h m,h 2m,h 3m,...} 4: for 1 i m, 1 j m 2 do 5: B i,j [x i+j m 1 ] h i 6: A i,j [x im j ] h (i 1)m 7: end for 8: C AB T 9: for 1 i < n do 10: b i C i /i (C i is the ith entry of C read rowwise) 11: end for 12: return b 1 +b 2 x +...+b n 1 x n 1

17 Comparison with BK 2.1 Let m = n. Fast Lagrange inversion for reversion: 1. 2m + O(1) polynomial multiplications 2. One (m m 2 ) times (m 2 m) matrix multiplication 3. O(n) additional operations Fast Horner s rule (BK 2.1) for composition: 1. m polynomial multiplications, each with cost M(n) 2. One (m m) times (m m 2 ) matrix multiplication 3. m polynomial multiplications and additions Both O(n 1/2 (M(n)+MM(n 1/2 ))), same constant factor.

18 Speedup by avoiding Newton iteration Suppose composition has cost C(n) cn r. Overhead of reversion using Newton iteration: C(n)+C(n/2)+C(n/4)+... = cn r ( 2 r 2 r 1 Classical polynomial multiplication: 4 31 (8+ 2) FFT polynomial multiplication, classical matrix multiplication: 4/3 )

19 Newton iteration speedup, visually speedup r

20 Faster matrix multiplication The matrices in BK 2.1 are full, whereas the matrix A in fast Lagrange inversion is half empty. Can we exploit this structure while using fast matrix multiplication (ω < 3)?

21 Faster matrix multiplication, first algorithm Decompose each m m block A i into blocks of size (m/k) (m/k) such that only the bottom row in is nonempty. Asymptotic speedup s: s = m ω+1 / k=1 ( m k m ) k 2( m ) ω k +1 k s > ( k=0 ) 1 2 k 1 2 kω = ω > 1

22 Faster matrix multiplication, second algorithm Write AB T = (AP)(P 1 B T ) where P is a permutation matrix that makes each m m block in A lower triangular. Recursive triangular multiplication: R(k) = 4R(k/2)+2(k/2) ω +O(k 2 ), s = 2 ω 1 2. s > 1 when ω > log , and better than the first method when ω > 1+log 2 (2+ 2) Optimal factor-two speedup with classical multiplication, and 3/2 speedup with Strassen.

23 Matrix multiplication speedup 2.0 s Ω

24 Fast rectangular matrix multiplication MM(x,y,z): (x y) by (y z) matrix multiplication Huang and Pan (1998): MM(m,m,m 2 ) = O(n ) < mmm(m,m,m) = O(n ) The exponents of MM(m,m 2,m) and MM(m,m,m 2 ) are the same, but could we lose a constant factor? Conjecture: MM(m,m 2,m) = (1+o(1))MM(m,m,m 2 )

25 Total speedup Theoretical speedup of fast Lagrange inversion over BK 2.1 by both avoiding Newton iteration and speeding up matrix multiplication. Dominant operation Complexity Newton Matrix Total Polynomial, classical O(n 5/2 ) Polynomial, Karatsuba O(n 1/2+log 2 3 ) Matrix, classical O(n 2 ) Matrix, Strassen O(n (1+log 2 7)/2 ) Matrix, Cop.-Win. O(n ) Matrix, Huang-Pan O(n ) ? 1.458? (Polynomial, FFT) O(n 3/2 log 1+o(1) n)

26 Implementation Fast composition and reversion implemented in FLINT ( Rings: Z, Z/pZ, Q

27 Reversion over (Z/pZ)[[x]] Input is a a power series with random coefficients, p = n Lagrange BK 2.1 Fast Lagrange µs 21 µs 7.0 µs ms 1.3 ms 0.76 ms ms 100 ms 62 ms s 5.1 s 3.3 s h 260 s 160 s Observed speedup: 1.6 Predicted speedup with O(n 1+o(1) ) multiplication and O(n 2 ) matrix multiplication: 1.54 Matrix multiplication negligible (< 10% of running time)

28 Reversion over Z[[x]] f 1 (x) = k 1k!x k f 2 (x) = x 1 4x f 3 (x) = x +x2 1+x +x 2 n Lagrange BK 2.1 Fast Lagrange f 1 f 2 f 3 f 1 f 2 f 3 f 1 f 2 f µs ms ms s s s Fast Lagrange inversion can lose its advantage (but usually does not get significantly slower than BK 2.1) It can also be much faster (if x/f(x) is small) Matrix multiplication is negligible (< 10% of running time)

29 Reversion over Q[[x]] f 4 (x) = exp(x) 1 f 5 (x) = x exp(x) f 6 (x) = 3x(1 x2 ) 2(1 x +x 2 ) 2 n Lagrange BK 2.1 Fast Lagrange f 4 f 5 f 6 f 4 f 5 f 6 f 4 f 5 f µs ms ms s s Matrix multiplication is negligible (< 20% of running time), but 10 times more costly if not implemented carefully (clearing denominators)!

30 Conclusions Reversion can be done using an algorithm analogous to BK 2.1, but directly (without Newton iteration) Faster than classical algorithms (no overhead), faster than BK 2.1 Faster than BK 2.2 up to fairly large n (since matrix multiplication is cheap in practice) But memory consumption can be a problem (BK 2.2 asymptotically better)

Fast reversion of formal power series

Fast reversion of formal power series Fredrik Johansson LFANT, INRIA Bordeaux RAIM, 2016-06-29, Banyuls-sur-mer 1 / 30 Reversion of power series F = exp(x) 1 = x + x 2 2! + x 3 3! + x 4 G = log(1 + x)