Fast reversion of formal power series

Size: px

Start display at page:

Download "Fast reversion of formal power series"

Elizabeth Carpenter
6 years ago
Views:

1 Fast reversion of formal power series Fredrik Johansson LFANT, INRIA Bordeaux RAIM, , Banyuls-sur-mer 1 / 30

2 Reversion of power series F = exp(x) 1 = x + x 2 2! + x 3 3! + x 4 G = log(1 + x) = x x x 3 3 x 4 4! F (G(x)) = G(F (x)) = x, G = F 1 and F = G 1 Applications: Combinatorics, recurrences (example: counting binary trees) Numerical approximation of F 1 Asymptotic analysis 2 / 30

3 Operations in R[[x]]/ x n n 1 F = F (x) = f k x k, Addition k=0 Multiplication [x k ]F (x) = f k R F + G = n 1 k=0 (f k + g k )x k FG = n 1 k=0 ( k i=0 f ig k i ) x k Reciprocal If f 0 is a unit, find G such that FG = 1 Composition Reversion If g 0 = 0, F (G) = n 1 k=0 f kg k If f 0 = 0 and f 1 is a unit, find G s.t. F (G) = x 3 / 30

4 Computational complexity Counting operations in the base ring R. M(n): multiplying two length-n polynomials Classical M(n) = O(n 2 ) Karatsuba M(n) = O(n log 2 (3) ) = O(n 1.59 ) Fast Fourier Transform M(n) = O(n log 1+o(1) n) = O(n 1+o(1) ) MM(n) = O(n ω ): multiplying two n n matrices Classical MM(n) = O(n 3 ) Strassen MM(n) = O(n log 2 (7) ) = O(n 2.81 ) Coppersmith-Winograd-Le Gall MM(n) = O(n ) 4 / 30

5 Newton iteration for power series If ϕ(g k ) = 0 mod x n, then (under natural assumptions) ϕ(g k+1 ) = 0 mod x 2n, G k+1 = G k ϕ(g k) ϕ (G k ) O(M(n)) computation of 1/F : take ϕ(g) = 1 G F, giving: G k+1 = 2G k FG 2 k. 5 / 30

6 Reversion using Newton iteration Given F = f 0 + f 1 x +... with f 0 = 0 and f 1 invertible. Take ϕ(g) = F (G) x, giving: G k+1 = G k F (G) x F. (G) Brent and Kung, 1978: up to constant factors: composition reversion 6 / 30

7 Fast composition: elementary functions Assume that 1, 2,..., n 1 are invertible in R. n 2 F = (k + 1)f k+1 x k, k=0 log(1 + F ) = n 1 ( ) 1 F = f k 1 x k k k=1 F 1 + F, atan(f ) = F 1 + F 2 Newton iteration: log exp, atan tan. sin, cos, sinh, cosh, asin... using algebraic transformations. Cost: O(M(n)) = O(n 1+o(1) ). Generalization: differential equations (hypergeometric,...) 7 / 30

8 Composition of general power series Horner s rule F (G) = (... (f n 1 G + f n 2 ) G f 1 ) G + f 0 Complexity: O(nM(n)) Brent-Kung 2.1: baby-step, giant-step version of Horner s rule Complexity: O(n 1/2 M(n) + n 1/2 MM(n 1/2 )) Brent-Kung 2.2: divide-and-conquer Taylor expansion Complexity: O((n log n) 1/2 M(n)) 8 / 30

9 Brent-Kung algorithm 2.1 Example with n = 9, m = n = 3 Computing F (G) where F = f 0 + f 1 x f 8 x 8 (f 0 + f 1 G + f 2 G 2 ) + (f 3 + f 4 G + f 5 G 2 )G 3 + (f 6 + f 7 G + f 8 G 2 )G 6 (m m) (m m 2 ) matrix multiplication: f 0 f 1 f 2 1 f 3 f 4 f 5 G f 6 f 7 f 8 G 2 Final combination using Horner s rule. 9 / 30

10 Complexity of composition algorithms Horner s rule: O(nM(n)) BK 2.1: O(n 1/2 M(n) + n 1/2 MM(n 1/2 )) BK 2.2: O((n log n) 1/2 M(n)) = O(n 3/2+o(1) ) M(n) MM(n) Horner BK 2.1 BK 2.2 n 2 n 3 n 3 n 2.5 n 2.5 log 0.5 n n 1.59 n 3 n 2.59 n 2.09 n 2.09 log 0.5 n n log n n 3 n 2 log n n 2 n 1.5 log 1.5 n n log n n n 2 log n n n 1.5 log 1.5 n (n log n) (n 2 ) (n 2 log n) (n 1.5 log n) (n 1.5 log 1.5 n) BK 2.1 wins for small n, BK 2.2 wins for large n BK 2.1 wins with hypothetical O(n 2 ) matrix multiplication 10 / 30

11 The Lagrange inversion theorem The coefficients of F 1 are given by [x k ]F 1 = 1 ( x ) k k [x k 1 ]. F O(nM(n)) algorithm: compute H = x/f using Newton iteration for the reciprocal, then compute H, H 2, H 3,..., H n 1 Better than classical algorithms (O(n 3 )) Similar to Newton iteration + Horner s rule Worse than Newton iteration + BK2.1 or BK / 30

12 Fast Lagrange inversion We can extract a single coefficient in H a+b = H a H b using O(n) operations: [x k ]H a+b = k ( [x i ]H a) ([x k i ]H b) i=0 Compute H, H 2,..., H m 1, then H m, H 2m, H 3m,..., H n/m m. Choose m n need O( n) polynomial multiplications, plus O(n 2 ) coefficient operations. 12 / 30

13 Fast Lagrange inversion, first version 1: m n 2: H x/f 3: for 1 i < m do 4: H i+1 H i H 5: b i 1 i [x i 1 ]H i 6: end for 7: T H m 8: for i = m, 2m, 3m,..., lm < n do 9: b i 1 i [x i 1 ]T 10: for 1 j < m while i + j < n do 11: b i+j 1 i+j 12: end for 13: T T H m i+j 1 k=0 ([x k ]T ) ([x i+j k 1 ]H j ) 14: end for 15: return b 1 x + b 2 x b n 1 x n 1 13 / 30

14 Using matrix multiplication Example: n = 8, m = 8 1 = 3. We want the coefficients b 1, b 2,..., b 7 of 1, x,..., x 6 in the respective powers of H. h k,i = [x i ]H k h 0,2 h 0,1 h 0, A = h 3,5 h 3,4 h 3,3 h 3,2 h 3,1 h 3, h 6,6 h 6,5 h 6,4 h 6,3 h 6,2 h 6,1 h 6,0 0 0 h 1,0 h 1,1 h 1,2 h 1,3 h 1,4 h 1,5 h 1,6 B = 0 h 2,0 h 2,1 h 2,2 h 2,3 h 2,4 h 2,5 h 2,6 h 3,0 h 3,1 h 3,2 h 3,3 h 3,4 h 3,5 h 3,6 b 1 b 2 b 3 AB T = b 4 b 5 b 6 b 7 14 / 30

15 Fast Lagrange inversion, matrix version 1: m n 1 2: H x/f 3: {Assemble m m 2 matrices B and A from H, H 2,..., H m and H m, H 2m, H 3m,...} 4: for 1 i m, 1 j m 2 do 5: B i,j [x i+j m 1 ] H i 6: A i,j [x im j ] H (i 1)m 7: end for 8: C AB T 9: for 1 i < n do 10: b i C i /i (C i is the ith entry of C read rowwise) 11: end for 12: return b 1 x + b 2 x b n 1 x n 1 15 / 30

16 Comparison with BK 2.1 Let m = n. Fast Lagrange inversion for reversion: 1. 2m + O(1) polynomial multiplications 2. One (m m 2 ) times (m 2 m) matrix multiplication 3. O(n) additional operations Fast Horner s rule (BK 2.1) for composition: 1. m polynomial multiplications, each with cost M(n) 2. One (m m) times (m m 2 ) matrix multiplication 3. m polynomial multiplications and additions Both O(n 1/2 (M(n) + MM(n 1/2 ))), same constant factor. 16 / 30

17 Speedup by avoiding Newton iteration If composition costs C(n) cn r, reversion via Newton costs ( ) 2 C(n) + C(n/2) + C(n/4) +... = cn r r 2 r 1 speedup r Classical polynomial multiplication: 4 31 (8 + 2) FFT polynomial multiplication, classical matrix multiplication: 4/3 17 / 30

18 Faster matrix multiplication Brent-Kung 2.1: Fast Lagrange inversion: = = Can we exploit the structure of A when ω < 3? 18 / 30

19 Faster matrix multiplication, first algorithm Decompose each m m block of A into (m/k) (m/k) blocks. Asymptotic speedup s: s = m ω+1 ( ) m k=1 k m k+1 k ( ) 2 m ω > k 1 k=0 2k 1 2 kω = ω > 1 19 / 30

20 Faster matrix multiplication, second algorithm Write AB T = (AP)(P 1 B T ) where P is a permutation matrix that makes each m m block in A lower triangular. Recursive triangular multiplication: R(k) = 4R(k/2) + 2(k/2) ω + O(k 2 ) Speedup: s = 2 ω 1 2. Note: s > 1 when ω > log / 30

21 Matrix multiplication speedup Speedup Alg.1 Alg ω Crossover at ω = 1 + log 2 (2 + 2) s = 2 with classical matrix multiplication (ω = 3) s = 1.5 with Strassen multiplication s = with Coppersmith-Winograd-Le Gall 21 / 30

22 Using fast rectangular matrix multiplication MM(x, y, z): (x y) by (y z) matrix multiplication Huang and Pan (1998): MM(m, m, m 2 ) = O(n ) < mmm(m, m, m) = O(n ) By a transposition argument, MM(m, m 2, m) = (1 + o(1))mm(m, m, m 2 ) Open problem: can we get a speedup > 1? 22 / 30

23 Total speedup Theoretical speedup of fast Lagrange inversion over BK 2.1 by both avoiding Newton iteration and speeding up matrix multiplication. Dominant operation Complexity Newton Matrix Total Polynomial, classical O(n 5/2 ) Polynomial, Karatsuba O(n 1/2+log 2 3 ) Matrix, classical O(n 2 ) Matrix, Strassen O(n (1+log 2 7)/2 ) Matrix, Cop.-Win.-LG O(n ) Matrix, Huang-Pan O(n ) ? 1.458? (Polynomial, FFT) O(n 3/2 log 1+o(1) n) / 30

24 Implementation Composition and reversion implemented in FLINT: Z/pZ, p < 2 64 Z Q In Arb (interval coefficients): R C 24 / 30

25 Reversion over Z/pZ (p = ) n Lagrange BK 2.1 BK 2.2 Fast Lagrange (8%) (20%) (28%) Time in seconds. (Relative time spent on matrix multiplication.) 25 / 30

26 Reversion over Z/pZ (p = ) Relative time Lagrange BK 2.2 BK 2.1 Fast Lagrange n 26 / 30

27 Reversion over Z F 1 = k 1 k!x k, F 2 = x, F 3 = x + x 2 1 4x 1 + x + x 2 n BK 2.1 Fast Lagrange F 1 F 2 F 3 F 1 F 2 F Time in seconds. 27 / 30

28 Reversion over Q F 1 = exp(x) 1, F 2 = x exp(x), F 3 = 3x(1 x 2 ) 2(1 x + x 2 ) 2 n BK 2.1 Fast Lagrange F 1 F 2 F 3 F 1 F 2 F Time in seconds. 28 / 30

29 Reversion over R F 1 = exp(x) 1, F 2 = log(1 + x), F 3 = Γ(1 + x) 1 n BK 2.1 Fast Lagrange F 1 F 2 F 3 F 1 F 2 F Precision doubled until the coefficient of x n 1 is a precise interval. Time in seconds. 29 / 30

30 Conclusion and questions Reversion algorithm analogous to BK 2.1, avoiding Newton iteration Usually the fastest algorithm for reversion in practice. Is there a Newton-free analog of BK 2.2 for reversion? Can more be said about the structured matrix products? Can denominators be handled better? Numerical stability? Note: the algorithm has been published in FJ, A fast algorithm for reversion of power series, Math. Comp. 84, , Some more comments in my PhD thesis. 30 / 30

Fast reversion of power series

Fast reversion of power series Fredrik Johansson November 2011 Overview Fast power series arithmetic Fast composition and reversion (Brent and Kung, 1978) A new algorithm for reversion Implementation results