Black Box Linear Algebra

Transcription

1 Black Box Linear Algebra An Introduction to Wiedemann s Approach William J. Turner Department of Mathematics & Computer Science Wabash College

2 Symbolic Computation Sometimes called Computer Algebra Symbols or exact arithmetic Not floating point numbers (numerical analysis) No round off errors Algorithms may not be compatible Numerical Analysis Find approximation quickly May never find exact solution Symbolic Computation Find exact solution quickly May never approximate Black Box Linear Algebra p.1/29

3 Probabilistic Algorithms Three types: Monte Carlo: Always fast, probably correct Las Vegas: Probably fast, always correct Certificate BPP: Bounded Probabalistic Polynomial Time Probably fast, probably correct Atlantic City? Schwartz-Zippel Lemma Probability randomly choose root of polynomial Black Box Linear Algebra p.2/29

4 Black Box Linear Algebra p.3/29 Solving Linear Systems

5 Black Box Linear Algebra p.4/29 Gaussian Elimination Row echelon form and back substitution Must know how matrix stored Matrix changed in calculation Sparse matrices may become dense

6 Black Box Matrix Model External view of matrix Only matrix-vector products allowed Independent of implementation Implementations efficient in time or space Not unique to symbolic computation Can compute Krylov sequence Black Box Linear Algebra p.5/29

7 Black Box Linear Algebra p.6/29 Examples Matrix storage time Arbitrary matrix Sparse matrix nonzero entries) ( Hilbert matrix ) ( Toeplitz matrix

8 Black Box Linear Algebra p.7/29 Linearly Generated Sequences. Let be a vector space over field A sequence and is linearly generated if and only if there exist such that for all or

9 Black Box Linear Algebra p.8/29 Generating Polynomials generates The polynomial -module is an

10 Minimal Generating Polynomial If and, then is an ideal is a principal ideal domain There exists a unique monic generator of minimal degree, the minimal generating polynomial of sequence Minimal polynomial divides all generating polynomials Black Box Linear Algebra p.9/29

11 Black Box Linear Algebra p.10/29 Example: Fibonacci Numbers Minimal polynomial is also generates also generates elements of Skips first

12 Black Box Linear Algebra p.11/29 Matrix Power Sequence generates Cayley-Hamilton Theorem be minimal polynomial of Let

13 Krylov Sequence Let be minimal polynomial of Black Box Linear Algebra p.12/29

14 Solving, and, Suppose satisfies requirements If exists, then is a solution Black Box Linear Algebra p.13/29

15 Black Box Linear Algebra p.14/29 Nonsingular is the unique solution

16 Bilinear Projection Sequence T T T T T be minimal polynomial of Let with, then If chosen randomly from finite probability at least Certificate: Black Box Linear Algebra p.15/29

17 Computing Minimal Polynomial Must know Extended Euclidean Algorithm Inputs are and Stop when Minimal polynomial is reversal of Berlekamp-Massey Algorithm From coding theory Interpolates elements of scalar sequence Related to Extended Euclidean Algorithm Black Box Linear Algebra p.16/29

18 Black Box Linear Algebra p.17/29 Berlekamp-Massey Algorithm with generator with Input: Scalar sequence Output: Minimal polynomial of sequence Initial guess 1: generates do to 2: for 3: 4: does not generate then 5: if 6: update to generate 7: end if 8: end for

19 Black Box Linear Algebra p.18/29 Example: Fibonacci Numbers recursion relation

20 Black Box Linear Algebra p.19/29 Wiedemann s Algorithm T Only need and Require: nonsingular such that Ensure: where random vector in 1: 2: use Berlekamp-Massey to compute Store only 3:

21 Black Box Linear Algebra p.20/29 Back to original problem

22 Back to original problem T Black Box Linear Algebra p.21/29

23 Singular Kaltofen and Saunders (1991): If known leading principal minor nonzero Randomly choose Solve is leading is first nonsingular system rows of submatrix of Then uniformly samples solution manifold Black Box Linear Algebra p.22/29

24 Precondition Matrix Want leading principal minor of nonzero or Need efficient matrix-vector product New linear system Black Box Linear Algebra p.23/29

25 Generic Rank Profile Wiedemann (1986):, parameterized and can realize any permutation Kaltofen and Saunders (1991):, unit upper triangular Toeplitz and unit lower triangular Toeplitz Chen et al. (2002); Turner (2002):,, based on butterfly networks Generic exchange matrix mixes inputs Turner (2002): Generalize butterfly networks to radix- switches Patton and Turner (2004): Toeplitz (or Hankel) matrix switches Black Box Linear Algebra p.24/29

26 Matrix Rank Kaltofen and Saunders (1991): If (unknown) Leading principal minors nonzero up to Then with probability at least Black Box Linear Algebra p.25/29

27 Matrix Minimal Polynomial If chosen randomly, then with probability at least If and chosen randomly, then least with probability at Black Box Linear Algebra p.26/29

28 Rank Preconditioners Generic rank profile preconditioner and diagonal matrix Chen et al. (2002): and Toeplitz unit lower triangular Toeplitz, upper triangular Turner (2002, 2003): Relax slightly generic rank profile Patton and Turner (2004):... Toeplitz (or Hankel) Patton and Turner (2004): Black Box Linear Algebra p.27/29

29 Black Box Linear Algebra p.28/29 Matrix Determinant If Then

30 Determinant Preconditioners Generic rank profile preconditioner and diagonal matrix Kaltofen and Pan (1992): unit upper triangular Toeplitz and lower triangular Toeplitz Chen et al. (2002):... Turner (2002, 2003): Black Box Linear Algebra p.29/29

31 References E. R. BERLEKAMP (1968). Algebraic Coding Theory. McGraw-Hill, New York. L. CHEN, W. EBERLY, E. KALTOFEN, B. D. SAUNDERS, W. J. TURNER, and G. VILLARD (2002). Efficient Matrix Preconditioners for Black Box Linear Algebra. Linear Algebra and its Applications, : Special issue on Infinite Systems of Linear Equations Finitely Specified. J. L. DORNSTETTER (1987). On the Equivalence Between Berlekamp s and Euclid s Algorithms. IEEE Transactions on Information Theory, IT-33(3): J. VON ZUR GATHEN and J. GERHARD (1999). Modern Computer Algebra. Cambridge University Press, Cambridge. E. KALTOFEN (1992). Efficient Solution of Sparse Linear Systems. Lecture notes, Computer Science Department, Rensselaer Polytechnic Institute, Troy, N.Y. E. KALTOFEN and V. PAN (1991). Processor Efficient Parallel Solution of Linear Systems over an Abstract Field. In Proceedings of SPAA 91 3rd Annual ACM Symposium on Parallel Algorithms and Architectures, ACM Press, New York, New York. E. KALTOFEN and V. PAN (1992). Processor Efficient Parallel Solution of Linear Systems II: the Positive Characteristic and Singular Cases. In Proceedings of the 33rd Annual Symposium on Foundations of Computer Science, IEEE Computer Society Press, Los Alamitos, California. E. KALTOFEN and B. D. SAUNDERS (1991). On Wiedemann s Method of Solving Sparse Linear Systems. In H. F. MATTSON, T. MORA, and T. R. N. RAO (eds.), AAECC-9: Proceedings of the 1991 Applied Algebra, Algebraic Algorithms and Error- Correcting Codes, International Conference, vol. 539 of Lecture Notes in Computer Science, Springer Verlag, Heidelberg, Germany. J. L. MASSEY (1969). Shift-Register Synthesis and BCH Decoding. IEEE Transactions on Information Theory, IT-15(1): A. E. PATTON and W. J. TURNER (2004). Toeplitz and Hankel Matrix Preconditioners for Black Box Linear Algebra. To be submitted to International Symposium on Symbolic and Algebraic Computation J. T. SCHWARTZ (1980). Fast Probabilistic Algorithms for Verification of Polynomial Identities. J. ACM, 27: W. J. TURNER (2002). Black Box Linear Algebra with the LinBox Library. Ph.D. thesis, North Carolina State University, Raleigh, North Carolina. W. J. TURNER (2003). Determinantal Divisors and Matrix Preconditioners. Submitted to Journal of Symbolic Computation. D. H. WIEDEMANN (1986). Solving Sparse Linear Equations Over Finite Fields. IEEE Transactions on Information Theory, IT-32(1): R. ZIPPEL (1979). Probabilistic Algorithms for Sparse Polynomials. In E. W. NG (ed.), Symbolic and Algebraic Computation, EUROSAM 79, An International Symposiumon Symbolic and Algebraic Computation, Marseille, France, June 1979, Proceedings, vol. 72 of Lecture Notes in Computer Science, Springer Verlag, Heidelberg, Germany. R. ZIPPEL (1990). Interpolating Polynomials from Their Values. Journal of Symbolic Computation, 9(3):