Exact Arithmetic on a Computer

Size: px

Start display at page:

Download "Exact Arithmetic on a Computer"

Molly Jones
5 years ago
Views:

1 Exact Arithmetic on a Computer Symbolic Computation and Computer Algebra William J. Turner Department of Mathematics & Computer Science Wabash College Crawfordsville, IN Tuesday 21 September 2010 W. J. Turner Exact Arithmetic on a Computer 1/ 23

2 Outline 1 Introduction Symbolic Computation Symbolic Computation vs. Numerical Analysis Symbolic Algorithms 2 Fundamental Algorithms Storing Integers and Polynomials Classical Arithmetic Algorithms 3 Fast Multiplication 4 Division 5 Solving Polynomial Equations W. J. Turner Exact Arithmetic on a Computer 2/ 23

3 Symbolic Computation Superset of computer algebra Symbols or exact arithmetic Exact finite representation of mathematical structures Abstract structures (groups, rings, fields, etc.) Polynomials and power series Linear algebra Number theory Algebraic geometry Computational group theory Differential equations Automated theorem proving W. J. Turner Exact Arithmetic on a Computer 3/ 23

4 Computer Algebra Systems General Purpose Systems AXIOM, Magma, Maple, Mathematica, REDUCE, SAGE Special Purpose Systems CoCoA (Computations in Commutative Algebra) GAP (Groups, Algorithms, and Programming) NTL (Number Theory Library) Singular (polynomial computations) Theorema (automated theorem proving) W. J. Turner Exact Arithmetic on a Computer 4/ 23

5 Computer Algebra Systems General Purpose Systems AXIOM, Magma, Maple, Mathematica, REDUCE, SAGE Special Purpose Systems CoCoA (Computations in Commutative Algebra) GAP (Groups, Algorithms, and Programming) NTL (Number Theory Library) Singular (polynomial computations) Theorema (automated theorem proving) W. J. Turner Exact Arithmetic on a Computer 4/ 23

6 Long History Ancient Algorithms Euclidean Algorithm Chinese Remainder Algorithm Isaac Newton s The Universal Arithmetic (1728) Systematically discusses rules for manipulating universal mathematical expressions, that is, formulae containing symbolic indeterminates, and algorithms for solving equations built with these expressions. W. J. Turner Exact Arithmetic on a Computer 5/ 23

7 Long History Ancient Algorithms Euclidean Algorithm Chinese Remainder Algorithm Isaac Newton s The Universal Arithmetic (1728) Systematically discusses rules for manipulating universal mathematical expressions, that is, formulae containing symbolic indeterminates, and algorithms for solving equations built with these expressions. W. J. Turner Exact Arithmetic on a Computer 5/ 23

8 Symbolic Computation vs. Numerical Analysis Numerical Analysis Floating point numbers (approximate real values) Find approximation quickly Error propagation important Condition Number Stability Symbolic Computation Find exact solution quickly May never approximate Structure may not have a metric Algorithms may not be compatible Numerical algorithms may never find exact solution Symbolic algorithms may be ill-conditioned or unstable W. J. Turner Exact Arithmetic on a Computer 6/ 23

9 Symbolic Computation vs. Numerical Analysis Numerical Analysis Floating point numbers (approximate real values) Find approximation quickly Error propagation important Condition Number Stability Symbolic Computation Find exact solution quickly May never approximate Structure may not have a metric Algorithms may not be compatible Numerical algorithms may never find exact solution Symbolic algorithms may be ill-conditioned or unstable W. J. Turner Exact Arithmetic on a Computer 6/ 23

10 Symbolic Computation vs. Numerical Analysis Numerical Analysis Floating point numbers (approximate real values) Find approximation quickly Error propagation important Condition Number Stability Symbolic Computation Find exact solution quickly May never approximate Structure may not have a metric Algorithms may not be compatible Numerical algorithms may never find exact solution Symbolic algorithms may be ill-conditioned or unstable W. J. Turner Exact Arithmetic on a Computer 6/ 23

11 Infinite Mathematical Structure General Approach Computer has finite memory Cannot compute exactly over reals, rationals, integers, etc. Compute bound M on desired solution Solve via modular algorithms & reconstruct solution Chinese Remainder Algorithm Hensel Lifting Rational Number Reconstruction Selecting the Modulus Big Prime Method: m = p Small Prime Method: m = i p i Small Prime Power Method: m = p l W. J. Turner Exact Arithmetic on a Computer 7/ 23

12 Infinite Mathematical Structure General Approach Computer has finite memory Cannot compute exactly over reals, rationals, integers, etc. Compute bound M on desired solution Solve via modular algorithms & reconstruct solution Chinese Remainder Algorithm Hensel Lifting Rational Number Reconstruction Selecting the Modulus Big Prime Method: m = p Small Prime Method: m = i p i Small Prime Power Method: m = p l W. J. Turner Exact Arithmetic on a Computer 7/ 23

13 Storing Integers and Polynomials Polynomials Integers Polynomials R[x] over ring R (e.g., Z m ) a = a n x n + a n 1 x n a 1 x + a 0 R[x] store degree n and coefficients a i for i = 0, 1, 2,..., n Radix r N >1 a = a n r n + a n 1 r n a 1 r + a 0 Z 0 a i < r for i = 0, 1, 2,..., n. store size n and digits a i for i = 0, 1, 2,..., n W. J. Turner Exact Arithmetic on a Computer 8/ 23

14 Storing Integers and Polynomials Polynomials Integers Polynomials R[x] over ring R (e.g., Z m ) a = a n x n + a n 1 x n a 1 x + a 0 R[x] store degree n and coefficients a i for i = 0, 1, 2,..., n Radix r N >1 a = a n r n + a n 1 r n a 1 r + a 0 Z 0 a i < r for i = 0, 1, 2,..., n. store size n and digits a i for i = 0, 1, 2,..., n W. J. Turner Exact Arithmetic on a Computer 8/ 23

15 Classical Addition Algorithm Polynomials n a = a i x i and b = i=0 Assume n = m : c = Algorithm for i = 0, 1, 2,... n do c i a i + b i end for Complexity O(n) ring operations m b i x i R[x] i=0 n c i = a + b = i=0 n (a i + b i )x i i=0 W. J. Turner Exact Arithmetic on a Computer 9/ 23

16 Classical Addition Algorithm Polynomials n a = a i x i and b = i=0 Assume n = m : c = Algorithm for i = 0, 1, 2,... n do c i a i + b i end for Complexity O(n) ring operations m b i x i R[x] i=0 n c i = a + b = i=0 n (a i + b i )x i i=0 W. J. Turner Exact Arithmetic on a Computer 9/ 23

17 Classical Addition Algorithm Polynomials n a = a i x i and b = i=0 Assume n = m : c = Algorithm for i = 0, 1, 2,... n do c i a i + b i end for Complexity O(n) ring operations m b i x i R[x] i=0 n c i = a + b = i=0 n (a i + b i )x i i=0 W. J. Turner Exact Arithmetic on a Computer 9/ 23

18 Classical Addition Algorithm Integer Algorithm c 0 0 for i = 0, 1, 2,... n do c i a i + b i + c i if c i r then c i c i r c i+1 1 else c i+1 0 end if end for Complexity O(n) word operations W. J. Turner Exact Arithmetic on a Computer 10/ 23

19 Classical Addition Algorithm Integer Algorithm c 0 0 for i = 0, 1, 2,... n do c i a i + b i + c i if c i r then c i c i r c i+1 1 else c i+1 0 end if end for Complexity O(n) word operations W. J. Turner Exact Arithmetic on a Computer 10/ 23

20 Classical Multiplication Algorithm Polynomial Algorithm Require: a = n i=0 a ix i and b = m i=0 b ix i for k = 0, 1, 2,... n + m do c k 0 for i = max{0, k m},..., min{n, k} do c k c k + a i b k i end for end for Complexity O(mn) ring operations n = m = O(n 2 ) ring operations W. J. Turner Exact Arithmetic on a Computer 11/ 23

21 Classical Multiplication Algorithm Polynomial Algorithm Require: a = n i=0 a ix i and b = m i=0 b ix i for k = 0, 1, 2,... n + m do c k 0 for i = max{0, k m},..., min{n, k} do c k c k + a i b k i end for end for Complexity O(mn) ring operations n = m = O(n 2 ) ring operations W. J. Turner Exact Arithmetic on a Computer 11/ 23

22 Classical Multiplication Algorithm Another Organization Require: a = n i=0 a ix i and b = m i=0 b ix i for i = 0, 1, 2,... n do d i a i x i b { x i just shifts a i by i places } end for return c n i=0 d i Integer Algorithm Require: a = ( 1) s n i=0 a ir i and b = ( 1) t m i=0 b ir i for i = 0, 1, 2,... n do d i a i r i b { r i just shifts a i by i places } end for return c ( 1) s+t n i=0 d i W. J. Turner Exact Arithmetic on a Computer 12/ 23

23 Classical Multiplication Algorithm Another Organization Require: a = n i=0 a ix i and b = m i=0 b ix i for i = 0, 1, 2,... n do d i a i x i b { x i just shifts a i by i places } end for return c n i=0 d i Integer Algorithm Require: a = ( 1) s n i=0 a ir i and b = ( 1) t m i=0 b ir i for i = 0, 1, 2,... n do d i a i r i b { r i just shifts a i by i places } end for return c ( 1) s+t n i=0 d i W. J. Turner Exact Arithmetic on a Computer 12/ 23

24 Classical Division with Remainder Algorithm Polynomial Synthetic Division Algorithm Require: a = n i=0 a ix i and b = m i=0 b ix i where b m is a unit and n m 0 Ensure: a = qb + r and deg r < m r a and u bm 1 for i = n m, n m 1,... 0 do if deg r = m + i then q i lc(r)u {Leading coefficient of r} r r q i x i b else q i 0 end if end for return q n m i=0 q ix i and r W. J. Turner Exact Arithmetic on a Computer 13/ 23

25 Fast Multiplication Roots of Unity Let R be a ring, n N >1, and ω R. ω is an nth root of unity if ω n = 1. ω is a primitive nth root of unity if 1 k < n = ω k 1. Discrete Fourrier Transform DFT ω : R n R n, f ( f (1), f (ω), f (ω 2 ),..., f (ω n 1 ) ) deg(f ) < n ω is primitive nth root of unity Fast Fourrier Transform (FFT) Can compute DFT recursively for n = 2 k O(n log n) ring operations W. J. Turner Exact Arithmetic on a Computer 14/ 23

26 Fast Multiplication Roots of Unity Let R be a ring, n N >1, and ω R. ω is an nth root of unity if ω n = 1. ω is a primitive nth root of unity if 1 k < n = ω k 1. Discrete Fourrier Transform DFT ω : R n R n, f ( f (1), f (ω), f (ω 2 ),..., f (ω n 1 ) ) deg(f ) < n ω is primitive nth root of unity Fast Fourrier Transform (FFT) Can compute DFT recursively for n = 2 k O(n log n) ring operations W. J. Turner Exact Arithmetic on a Computer 14/ 23

27 Fast Multiplication Roots of Unity Let R be a ring, n N >1, and ω R. ω is an nth root of unity if ω n = 1. ω is a primitive nth root of unity if 1 k < n = ω k 1. Discrete Fourrier Transform DFT ω : R n R n, f ( f (1), f (ω), f (ω 2 ),..., f (ω n 1 ) ) deg(f ) < n ω is primitive nth root of unity Fast Fourrier Transform (FFT) Can compute DFT recursively for n = 2 k O(n log n) ring operations W. J. Turner Exact Arithmetic on a Computer 14/ 23

28 Fast Multiplication DFT and Multiplication If deg(f ) + deg(g) < n, then DFT ω (f g) = DFT ω (f ) DFT ω (g). Fast Multiplication Algorithm Require: deg(f ), deg(g) < n k log 2 (2n) ω primitive 2 k th root of unity in R α DFT ω (f ) and β DFT ω (g) {via FFT} γ α β {pointwise multiplication} return DFT 1 ω (γ) = 1 n DFT ω 1(γ) Complexity O(n log n) ring operations O(n log n loglog n) ring operations if must extend ring W. J. Turner Exact Arithmetic on a Computer 15/ 23

29 Fast Multiplication DFT and Multiplication If deg(f ) + deg(g) < n, then DFT ω (f g) = DFT ω (f ) DFT ω (g). Fast Multiplication Algorithm Require: deg(f ), deg(g) < n k log 2 (2n) ω primitive 2 k th root of unity in R α DFT ω (f ) and β DFT ω (g) {via FFT} γ α β {pointwise multiplication} return DFT 1 ω (γ) = 1 n DFT ω 1(γ) Complexity O(n log n) ring operations O(n log n loglog n) ring operations if must extend ring W. J. Turner Exact Arithmetic on a Computer 15/ 23

30 Fast Multiplication DFT and Multiplication If deg(f ) + deg(g) < n, then DFT ω (f g) = DFT ω (f ) DFT ω (g). Fast Multiplication Algorithm Require: deg(f ), deg(g) < n k log 2 (2n) ω primitive 2 k th root of unity in R α DFT ω (f ) and β DFT ω (g) {via FFT} γ α β {pointwise multiplication} return DFT 1 ω (γ) = 1 n DFT ω 1(γ) Complexity O(n log n) ring operations O(n log n loglog n) ring operations if must extend ring W. J. Turner Exact Arithmetic on a Computer 15/ 23

31 Fast Division Polynomial Reversal The reversal of a polynomial a = n i=0 a ix i is ( ) 1 rev k (a) = x k a x When k = n, rev(a) = rev n (a) reverses the coefficients of a. Reversals and Division If deg(a) = n, deg(b) = m, and b(0) = 1, then deg(r) < m so rev m 1 (r) is a polynomial and rev n (a) rev n m (q) = rev m (b) rev n m (q) + x n m+1 rev m 1 (r) rev m (b) rev n m (q) (mod x n m+1 ) = rev n (a) rev m (b) 1 mod x n m+1 W. J. Turner Exact Arithmetic on a Computer 16/ 23

32 Fast Division Polynomial Reversal The reversal of a polynomial a = n i=0 a ix i is ( ) 1 rev k (a) = x k a x When k = n, rev(a) = rev n (a) reverses the coefficients of a. Reversals and Division If deg(a) = n, deg(b) = m, and b(0) = 1, then deg(r) < m so rev m 1 (r) is a polynomial and rev n (a) rev n m (q) = rev m (b) rev n m (q) + x n m+1 rev m 1 (r) rev m (b) rev n m (q) (mod x n m+1 ) = rev n (a) rev m (b) 1 mod x n m+1 W. J. Turner Exact Arithmetic on a Computer 16/ 23

33 Fast Division Polynomial Reversal The reversal of a polynomial a = n i=0 a ix i is ( ) 1 rev k (a) = x k a x When k = n, rev(a) = rev n (a) reverses the coefficients of a. Reversals and Division If deg(a) = n, deg(b) = m, and b(0) = 1, then deg(r) < m so rev m 1 (r) is a polynomial and rev n (a) rev n m (q) = rev m (b) rev n m (q) + x n m+1 rev m 1 (r) rev m (b) rev n m (q) (mod x n m+1 ) = rev n (a) rev m (b) 1 mod x n m+1 W. J. Turner Exact Arithmetic on a Computer 16/ 23

34 Newton Iteration Newton s Iteration from Calculus Require: φ(y), initial estimate y 0, and tolerance τ Ensure: φ(ȳ) < τ k 0 while φ(y k ) τ do y k+1 y k φ(y k) φ (y k ) = y k φ(y k ) ( φ (y k ) ) 1 end while return ȳ y k W. J. Turner Exact Arithmetic on a Computer 17/ 23

35 Algebra Not Analysis Formal Derivative Let R be a ring (commutative, with 1). For φ = n i=0 φ iy i D[y], where D is a ring, we define the formal derivative of φ by φ = n iφ i y i 1 i=0 Approximations Given a modulus m, measure how well b approximates a by the highest power of m such that a b (mod m). Example Let m = 2. Then 9 is a better approximation for 17 than 15 because 9 17 (mod 2 3 ) but (mod 2 3 ). W. J. Turner Exact Arithmetic on a Computer 18/ 23

36 Algebra Not Analysis Formal Derivative Let R be a ring (commutative, with 1). For φ = n i=0 φ iy i D[y], where D is a ring, we define the formal derivative of φ by φ = n iφ i y i 1 i=0 Approximations Given a modulus m, measure how well b approximates a by the highest power of m such that a b (mod m). Example Let m = 2. Then 9 is a better approximation for 17 than 15 because 9 17 (mod 2 3 ) but (mod 2 3 ). W. J. Turner Exact Arithmetic on a Computer 18/ 23

37 Algebra Not Analysis Formal Derivative Let R be a ring (commutative, with 1). For φ = n i=0 φ iy i D[y], where D is a ring, we define the formal derivative of φ by φ = n iφ i y i 1 i=0 Approximations Given a modulus m, measure how well b approximates a by the highest power of m such that a b (mod m). Example Let m = 2. Then 9 is a better approximation for 17 than 15 because 9 17 (mod 2 3 ) but (mod 2 3 ). W. J. Turner Exact Arithmetic on a Computer 18/ 23

38 Inversion Using Newton Iteration Choosing the Function Given f R[x] = D, want φ D[y] such that φ(f 1 ) = 0. Must be invertible Update φ (φ ) 1 without division The Function φ(y) = 1 y f φ (y) = 1 y 2 φ(y) ( φ (y) ) 1 = y + fy 2 y φ(y) ( φ (y) ) 1 = 2y fy 2 W. J. Turner Exact Arithmetic on a Computer 19/ 23

39 Inversion Using Newton Iteration Choosing the Function Given f R[x] = D, want φ D[y] such that φ(f 1 ) = 0. Must be invertible Update φ (φ ) 1 without division The Function φ(y) = 1 y f φ (y) = 1 y 2 φ(y) ( φ (y) ) 1 = y + fy 2 y φ(y) ( φ (y) ) 1 = 2y fy 2 W. J. Turner Exact Arithmetic on a Computer 19/ 23

40 Inversion Using Newton Iteration Choosing the Function Given f R[x] = D, want φ D[y] such that φ(f 1 ) = 0. Must be invertible Update φ (φ ) 1 without division The Function φ(y) = 1 y f φ (y) = 1 y 2 φ(y) ( φ (y) ) 1 = y + fy 2 y φ(y) ( φ (y) ) 1 = 2y fy 2 W. J. Turner Exact Arithmetic on a Computer 19/ 23

41 Inversion Using Newton Iteration Choosing the Function Given f R[x] = D, want φ D[y] such that φ(f 1 ) = 0. Must be invertible Update φ (φ ) 1 without division The Function φ(y) = 1 y f φ (y) = 1 y 2 φ(y) ( φ (y) ) 1 = y + fy 2 y φ(y) ( φ (y) ) 1 = 2y fy 2 W. J. Turner Exact Arithmetic on a Computer 19/ 23

42 Inversion Using Newton Iteration Choosing the Function Given f R[x] = D, want φ D[y] such that φ(f 1 ) = 0. Must be invertible Update φ (φ ) 1 without division The Function φ(y) = 1 y f φ (y) = 1 y 2 φ(y) ( φ (y) ) 1 = y + fy 2 y φ(y) ( φ (y) ) 1 = 2y fy 2 W. J. Turner Exact Arithmetic on a Computer 19/ 23

43 Inversion Using Newton Iteration Inversion Algorithm Require: f R[x] with f (0) = 1 and l N Ensure: g R[x] with fg 1 (mod x l ) g 0 1 r log 2 l for i = 1,..., r do g i ( 2g i 1 f gi 1) 2 rem x 2 i { truncates polynomial } end for return g r Complexity 3M(l) + l = O(M(l)) ring operations M(l) is multiplication time W. J. Turner Exact Arithmetic on a Computer 20/ 23

44 Inversion Using Newton Iteration Inversion Algorithm Require: f R[x] with f (0) = 1 and l N Ensure: g R[x] with fg 1 (mod x l ) g 0 1 r log 2 l for i = 1,..., r do g i ( 2g i 1 f gi 1) 2 rem x 2 i { truncates polynomial } end for return g r Complexity 3M(l) + l = O(M(l)) ring operations M(l) is multiplication time W. J. Turner Exact Arithmetic on a Computer 20/ 23

45 Fast Division with Remainder Fast Division Algorithm Require: a, b R[x] where b 0 is monic. Ensure: q, r R[x] such that a = qb + r and deg r < deg b if deg a < deg b then return q 0 and r a end if m deg a deg b c (rev deg b (b)) 1 mod x m+1 { Newton Iteration } q rev deg a (a) c rem x m+1 { truncates polynomial } return q rev m (q ) and r a b q Complexity 3M(m) + M(n) + O(n) ring operations deg b = n and deg a = m + n W. J. Turner Exact Arithmetic on a Computer 21/ 23

46 Fast Division with Remainder Fast Division Algorithm Require: a, b R[x] where b 0 is monic. Ensure: q, r R[x] such that a = qb + r and deg r < deg b if deg a < deg b then return q 0 and r a end if m deg a deg b c (rev deg b (b)) 1 mod x m+1 { Newton Iteration } q rev deg a (a) c rem x m+1 { truncates polynomial } return q rev m (q ) and r a b q Complexity 3M(m) + M(n) + O(n) ring operations deg b = n and deg a = m + n W. J. Turner Exact Arithmetic on a Computer 21/ 23

47 Generalized Newton Iteration Newton Iteration Require: φ D[y], p R, l N >0, g 0 R with φ(g 0 ) 0 (mod p) and φ (g 0 ) invertible modulo p, and s 0 such that s 0 φ (g 0 ) 1 (mod p) Ensure: g R with φ(g) 0 (mod p l ) and g g 0 (mod p) r log 2 l for i = 1,..., r 1 do g i (g i 1 φ(g i 1 ) s i 1 ) mod p 2i s i ( 2s i 1 φ (g i ) si 1 2 ) mod p 2 i end for return g g r 1 φ(g r 1 ) s r 1 mod p l W. J. Turner Exact Arithmetic on a Computer 22/ 23

48 Complexity Polynomial Ring (3n + 3/2)M(l) + O(nl) when D = R[x] p = x l = 2 k deg y φ = n and deg x φ < l Integers O(nM(l log p)) word operations when R = Z 0 < g 0 < p deg φ = n φ i < p l for i = 0, 1,..., n W. J. Turner Exact Arithmetic on a Computer 23/ 23

49 Complexity Polynomial Ring (3n + 3/2)M(l) + O(nl) when D = R[x] p = x l = 2 k deg y φ = n and deg x φ < l Integers O(nM(l log p)) word operations when R = Z 0 < g 0 < p deg φ = n φ i < p l for i = 0, 1,..., n W. J. Turner Exact Arithmetic on a Computer 23/ 23

Fast Polynomial Multiplication

Fast Polynomial Multiplication Marc Moreno Maza CS 9652, October 4, 2017 Plan Primitive roots of unity The discrete Fourier transform Convolution of polynomials The fast Fourier transform Fast convolution