The Seven Dwarfs of Symbolic Computation and the Discovery of Reduced Symbolic Models. Erich Kaltofen North Carolina State University google->kaltofen

Size: px

Start display at page:

Download "The Seven Dwarfs of Symbolic Computation and the Discovery of Reduced Symbolic Models. Erich Kaltofen North Carolina State University google->kaltofen"

Ashlie Burke
5 years ago
Views:

1 The Seven Dwarfs of Symbolic Computation and the Discovery of Reduced Symbolic Models Erich Kaltofen North Carolina State University google->kaltofen

2 Outline 2 The 7 Dwarfs of Symbolic Computation Discovery of sparse rational function models Solving overdetermined linear systems optimally

3 Berkeley 13 Dwarfs 3 A dwarf is an algorithmic method that captures a pattern of computation and communication 1. Dense Linear Algebra 8. Combinational Logic 2. Sparse Linear Algebra 9. Graph Traversal 3. Spectral Methods 10. Dynamic Programming 4. N-Body Methods 11. Backtrack and Branch-and-Bound 5. Structured Grids 12. Graphical Models 6. Unstructured Grids 13. Finite State Machines 7. MapReduce How about Logic Programming, Symbolic Computation?

4 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 4

5 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 2. Exact polynomial and differential algebra, including polynomial arithmetic and computation of canonical forms such as Gröbner bases 4

6 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 2. Exact polynomial and differential algebra, including polynomial arithmetic and computation of canonical forms such as Gröbner bases 3. Inverse symbolic problems such as sparse interpolation and curve and surface parameterization 4

7 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 2. Exact polynomial and differential algebra, including polynomial arithmetic and computation of canonical forms such as Gröbner bases 3. Inverse symbolic problems such as sparse interpolation and curve and surface parameterization 4. Hybrid symbolic-numeric computation 4

8 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 2. Exact polynomial and differential algebra, including polynomial arithmetic and computation of canonical forms such as Gröbner bases 3. Inverse symbolic problems such as sparse interpolation and curve and surface parameterization 4. Hybrid symbolic-numeric computation 5. Tarski s algebraic theory of real geometry 4

9 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 2. Exact polynomial and differential algebra, including polynomial arithmetic and computation of canonical forms such as Gröbner bases 3. Inverse symbolic problems such as sparse interpolation and curve and surface parameterization 4. Hybrid symbolic-numeric computation 5. Tarski s algebraic theory of real geometry 6. Computation of closed form solutions to, e.g., sums, integrals, differential equations 4

10 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 2. Exact polynomial and differential algebra, including polynomial arithmetic and computation of canonical forms such as Gröbner bases 3. Inverse symbolic problems such as sparse interpolation and curve and surface parameterization 4. Hybrid symbolic-numeric computation 5. Tarski s algebraic theory of real geometry 6. Computation of closed form solutions to, e.g., sums, integrals, differential equations 7. Rewrite rule systems (simplification and theorem proving) and computational group theory 4

11 My 7 Dwarfs of Symbolic Computation 1. Exact linear algebra including algorithms for integer lattices 2. Exact polynomial and differential algebra, including polynomial arithmetic and computation of canonical forms such as Gröbner bases 3. Inverse symbolic problems such as sparse interpolation and curve and surface parameterization 4. Hybrid symbolic-numeric computation 5. Tarski s algebraic theory of real geometry 6. Computation of closed form solutions to, e.g., sums, integrals, differential equations 7. Rewrite rule systems (simplification and theorem proving) and computational group theory Model Discovery and Verification:

12 Outline 5 The 7 Dwarfs of Symbolic Computation Discovery of sparse rational function models Solving overdetermined linear systems optimally

13 Model Discovery Example Linear or quadratic best fit? What if best model is 2.5x7 y x 2 y 9?

14 Sparse rational function models [Kaltofen, Yang, Zhi 07] 7 ζ 1,...,ζ n C f g (ζ 1,...,ζ n )+noise f,g C[x 1,...,x n ],GCD( f,g) = 1 By sampling black box, compute sparse representation t f j=1 ã j x d j,1 1 x d j,n n t g b k=1 k x e k,1 1 x e k,n n = f g, ã j 0, b k 0 Note: Terms are not known.

15 Sparse rational function models [Kaltofen, Yang, Zhi 07] 7 ζ 1,...,ζ n C f g (ζ 1,...,ζ n )+noise f,g C[x 1,...,x n ],GCD( f,g) = 1 By sampling black box, compute sparse representation t f j=1 ã j x d j,1 1 x d j,n n t g b k=1 k x e k,1 1 x e k,n n = f g, ã j 0, b k 0 Note: Terms are not known. ZNIPR algorithm: Sparsity by a numeric Zippel-Schwartz lemma Numeric noise in values by condition numbers in random structured matrices

16 Zippel Exact and Numerical Interpolation of Rational Functions Idea: 8 Variable by variable interpolation Term supports like [Kaltofen 1986], instead of [Zippel 1979] Early termination [cf. Kaltofen and Lee 2003] Structured Total Least Norm (STLN) method employed in numerical case [cf. Kaltofen, Yang, Zhi 2005, 2006] STLN method to decide the support of the numerator and denominator. STLN method to compute the coefficients corresponding to the support.

17 Ingredients of ZNIPR algorithm 9 Probabilistic analysis of exact method Numerical Zippel/Schwartz lemma Condition numbers of randomized matrices

18 Example (Exact Case) Given the black box of the rational function f/g 10 f = x x 1x 2 2, g = 2x x 2, and the degree bounds d = 4, ē = 4. Supppose f 1 = f(x 1,a) = b 1 x b 2x 1, g 1 = g(x 1,a) = b 3 x b 4. and D 2 = {1,x 2 2 }, Ē 2 = {1,x 2 }, where b 1,b 2,b 3,b 4 K \ {0}.

19 Example (Exact Case) Given the black box of the rational function f/g 10 f = x x 1x 2 2, g = 2x x 2, and the degree bounds d = 4, ē = 4. Supppose f 1 = f(x 1,a) = b 1 x b 2x 1, g 1 = g(x 1,a) = b 3 x b 4. and D 2 = {1,x 2 2 }, Ē 2 = {1,x 2 }, where b 1,b 2,b 3,b 4 K \ {0}. The sets of the possible terms of f and g are D 2 = {x 1,x 3 1,x 1x 2 2}, E 2 = {1,x 3 1,x3 1 x 2,x 2 }.

20 Example (Exact Case) f and g can be represented as 11 f = y 1 x 1 + y 2 x y 3 x 1 x 2 2, g = z 1 + z 2 x z 3 x 3 1 x 2 + z 4 x 2. Pick random points p 1, p 2 K and compute the values: γ l = f(pl 1,pl 2 ) K \ {0, }, l = 0,1,...,L 1. g(p l 1,pl 2 ) γ 0 γ 0 γ 0 γ 0 p 1 p 3 1 p 1 p 2 2 γ 1 γ 1 p 3 1 γ 1 p 3 1 p 2 γ 1 p 2 p 2 1 (p 3 1 )2 (p 1 p 2 2 )2 γ 2 γ 2 (p 3 1 )2 γ 2 (p 3 1 p 2) 2 γ 2 p p L 1 1 (p 3 1 )L 1 (p 1 p 2 2 )L 1 γ L 1 γ L 1 (p 3 1 )L 1 γ L 1 (p 3 1 p2 2 )L 1 γ L 1 p L y 1 y 2 y 3 1 = 2 z 3 z

21 Exact Probabilistic Analysis 12 Rank deficiency of G is 1 if L D i E i w.h.p. Using smaller L: Increment L by 1 until rank deficiency of G is 1.

22 Table of Exact Interpolation 13 Ex. Coeff.Range d f,d g n t f,t g mod N(ZEIPR) N(KY 07) 1 [ 10,10] 3,3 2 6, [ 10,10] 5,2 4 6, [ 20,20] 2,4 6 2, [ 20,20] 1,6 8 4, [ 30,30] 10,5 10 7, [ 10,10] 15, , [ 10,10] 20, , [ 30,30] 30, , [ 50,50] 50, , [ 10,10] 2, ,

23 Numerical Zippel/Schwartz Lemma Let 14 0 (α 1,...,α s ) Z[i][α 1,...,α s ], i = 1, ζ j = exp( 2πi P j ) C, P j Z 3 distinct prime numbers 1 j s [cf. Giesbrecht, Labahn, Lee 2007] Suppose (ζ 1,...,ζ s ) 0 (use algebraic lemma to enforce) Then for random integers R j with 1 R j < P j Expected value{ (ζ R 1 1,...,ζ R s s ) } 1.

24 Numerical Zippel/Schwartz Lemma Let 14 0 (α 1,...,α s ) Z[i][α 1,...,α s ], i = 1, ζ j = exp( 2πi P j ) C, P j Z 3 distinct prime numbers 1 j s [cf. Giesbrecht, Labahn, Lee 2007] Suppose (ζ 1,...,ζ s ) 0 (use algebraic lemma to enforce) Then for random integers R j with 1 R j < P j Expected value{ (ζ R 1 1,...,ζ R s s ) } 1. Useful for support pruning. Also need estimates on matrix condition numbers?

25 Condition numbers of random n n matrices Entries from standard Gaussian distribution: 15 Expected value{logκ 2 (G n )} < log(n) [Edelman 1988; Chen and Dongarra 2005]

26 Condition numbers of random n n matrices Entries from standard Gaussian distribution: 15 Expected value{logκ 2 (G n )} < log(n) [Edelman 1988; Chen and Dongarra 2005] Random discrete noise: κ 2 (A+R n ) = n O(1) with high probability [Spielman and Teng 2004; Tao and Vu 2007]

27 Condition numbers of random n n matrices Entries from standard Gaussian distribution: 15 Expected value{logκ 2 (G n )} < log(n) [Edelman 1988; Chen and Dongarra 2005] Random discrete noise: κ 2 (A+R n ) = n O(1) with high probability [Spielman and Teng 2004; Tao and Vu 2007]... one rarely encounters ill-conditioned matrices in practice

28 Approximate Polynomial GCD: Sylvester matrices a m a m a 0 a m a 1 a a m a m a 0 b n b n b 0 b n b 1 b b n b 0 16 approximate polynomial GCD (common root) for a m x m + +a 0 and b n x n + +b 0 Sylvester-structure preserving deformation to singularity

29 Root distribution 20 polynomials of degree 20 Uniformly distributed coefficients = small structured condition number x 1 1.5

30 Root distribution 20 polynomials of degree 20 Binomially distributed coefficients = large structured condition number

31 Conditioning of random projections [Kaltofen and Trager 1988; Kaltofen and Yang 2007] 19 t f j=1 a j x d j,1 1 (ξ 2 x 1 + η 2 ) d j,2 (ξ n x 1 + η n ) d j,n t g k=1 b k x e k,1 1 (ξ 2 x 1 + η 2 ) e k,2 (ξ n x 1 + η n ) e k,n, ξ i,η i S random large structured condition numbers of Sylvester matrices [Zippel 1979; ours here] t f j=1 a j x d j,1 1 ξ d j,2 2 ξ d j,n n t g k=1 b k x e k,1 1 ξ e k,2 2 ξ e k,n n, ξ i S random small structured condition numbers of Sylvester matrices Well-conditioning of arising Fourier matrices to be established.

32 Table of Numerical Interpolation 20 Ex. Random Noise d f,d g t f,t g n N error (ZNIPR) , 1 2, e , 2 3, e , 4 2, e , 2 10, e , 7 25, e , 3 15, e , 20 7, e , 30 6, e , 60 7, e , 80 6, e , 0 6, e 12

33 Outline 21 The 7 Dwarfs of Symbolic Computation Discovery of sparse rational function models Solving overdetermined linear systems optimally

34 Fast matrix multiplication 22 Strassen s [1969] O(n 2.81 ) matrix multiplication algorithm m 1 (a 1,2 a 2,2 )(b 2,1 b 2,2 ) m 2 (a 1,1 + a 2,2 )(b 1,1 + b 2,2 ) m 3 (a 1,1 a 2,1 )(b 1,1 + b 1,2 ) m 4 (a 1,1 + a 1,2 )b 2,2 ) a 1,1 b 1,1 + a 1,2 b 2,1 = m 1 + m 2 m 4 + m 6 m 5 a 1,1 (b 1,2 b 2,2 ) a 1,1 b 1,2 + a 1,2 b 2,2 = m 4 + m 5 m 6 a 2,2 (b 2,1 b 1,1 ) a 2,1 b 1,1 + a 2,2 b 2,1 = m 6 + m 7 m 7 (a 2,1 + a 2,2 )b 1,1 ) a 2,1 b 1,2 + a 2,2 b 2,2 = m 2 m 3 + m 5 m 7 Coppersmith and Winograd [1990]: O(n 2.38 ) Problems reducible to matrix multiplication: linear system solving, determinant [Bunch and Hopcroft 1974], characteristic polyn. [Keller-Gehrig 85, Pernet&Storjohann 07], rational canonical form [Giesbrecht 92], factoring in Z 2 [x] [Berlekamp 69, Kaltofen and Shoup 95]

35 Fast matrix multiplication 22 Strassen s [1969] O(n 2.81 ) matrix multiplication algorithm m 1 (a 1,2 a 2,2 )(b 2,1 b 2,2 ) m 2 (a 1,1 + a 2,2 )(b 1,1 + b 2,2 ) m 3 (a 1,1 a 2,1 )(b 1,1 + b 1,2 ) m 4 (a 1,1 + a 1,2 )b 2,2 ) a 1,1 b 1,1 + a 1,2 b 2,1 = m 1 + m 2 m 4 + m 6 m 5 a 1,1 (b 1,2 b 2,2 ) a 1,1 b 1,2 + a 1,2 b 2,2 = m 4 + m 5 m 6 a 2,2 (b 2,1 b 1,1 ) a 2,1 b 1,1 + a 2,2 b 2,1 = m 6 + m 7 m 7 (a 2,1 + a 2,2 )b 1,1 ) a 2,1 b 1,2 + a 2,2 b 2,2 = m 2 m 3 + m 5 m 7 Coppersmith and Winograd [1990]: O(n 2.38 ) Problems reducible to matrix multiplication: linear system solving, determinant [Bunch and Hopcroft 1974], characteristic polyn. [Keller-Gehrig 85, Pernet&Storjohann 07], rational canonical form [Giesbrecht 92], factoring in Z 2 [x]: n 1.5+o(1) [Umans 07, w/o FMM]

36 Matrix preconditioners Theorem [Kaltofen and Saunders 91] Let A K n n with r = rank(a), S K with S <. Probab.(the first r rows of 1 u 2 u 3... u n 0 1 u 2... u n u A 23 are lin. indep. u i S uniformly randomly) 1 r S

37 Matrix preconditioners Theorem [Kaltofen and Saunders 91] Let A K n n with r = rank(a), S K with S <. Probab.(the first r rows of 1 u 2 u 3... u n 0 1 u 2... u n u A 23 are lin. indep. u i S uniformly randomly) 1 r S Note: Toeplitz times vector costs O(nlogn) ops. (w. roots of unity)

38 Application: solving an overdetermined system Ax = b 24 n T. A B 0 = 0 n p With high probability, B has rank of A, so solve Bx = c = (T b) 1. Arithmetic cost: TA, T b: O(pn log(n)) solve Bx = c: O(p 3 ) or O(p 2.38 ) check Ax = b: O(pn) (T b) p For p = O( nlog(n)) or O((nlog(n)) 0.72 ) the cost is O(pnlog(n))

39 Application: solving an overdetermined system Ax = b 24 n T. A B 0 = 0 n p With high probability, B has rank of A, so solve Bx = c = (T b) 1. Arithmetic cost: TA, T b: O(pn log(n)) solve Bx = c: O(p 3 ) or O(p 2.38 ) check Ax = b: O(pn) (T b) p For p = O( nlog(n)) or O((nlog(n)) 0.72 ) the cost is O(pnlog(n)) (Previously: O(p 2 n) or O(p 1.38 n))

40 Application: solving an overdetermined system Ax = b 24 n T. A B 0 = 0 n p With high probability, B has rank of A, so solve Bx = c = (T b) 1. Arithmetic cost: TA, T b: O(pn log(n)) solve Bx = c: O(p 3 ) or O(p 2.38 ) check Ax = b: O(pn) (T b) p For p = O( nlog(n)) or O((nlog(n)) 0.72 ) the cost is O(pnlog(n)) Highly over/underdet. systems can be solved essentially optimally.

41 Cond. Number Distr. of Precond. 25 Bryc, Włodzimierz, Dembo, Amir, and Jiang, Tiefeng. Spectral measure of large random Hankel, Markov and Toeplitz matrices. Ann. Probab., 34(1):1 38, Kaltofen, Zhi Bryc Zhidong Bai Bingyu Li, Jack Silverstein The distributions are tricky (we so far have no separation from zero theorem)!

42 26 Danke schön! Thank you!

Fast algorithms for factoring polynomials A selection. Erich Kaltofen North Carolina State University google->kaltofen google->han lu

Fast algorithms for factoring polynomials A selection Erich Kaltofen North Carolina State University google->kaltofen google->han lu Overview of my work 1 Theorem. Factorization in K[x] is undecidable