Accurate and Efficient Matrix Computations with Totally Positive Generalized Vandermonde Matrices Using Schur Functions

Transcription

1 Accurate and Efficient Matrix Computations with Totally Positive Generalized Matrices Using Schur Functions Plamen Koev Department of Mathematics UC Berkeley Joint work with Prof. James Demmel Supported by NSF and DOE Bay Area Scientific Computing Day, March, 00

2 GOALS Accurate (Small relative error) and Efficient (O(n 3 ) or perhaps O(n p ), independent of condition number) Linear Algebra A Ax = b LDU from GENP, GEPP, GECP SVD Can t be done for general matrices, must be structured Certain sparsity patterns Cauchy... Goal of this talk: Accurate and Efficient Linear Algebra for Generalized Matrices

3 Small Small Forward Backward Type of Any GENP Error in Error in Matrix det(a) A minor GEPP GECP SVD Ax = b Ax = b Cauchy Totally Positive Cauchy Totally positive Polynomial Poly. Vand. Orth. poly. Generalized TP Generalized Totally Positive = Matrix with all minors > 0

4 OUTLINE Model of arithmetic Classical method for achieving the goals for simple examples The Björck-Pereyra Method for Matrices How and why it works? Application to TP Generalized matrices

5 How can we lose accuracy in computing in floating point? fl(a b) = (a b)( + δ) model of arithmetic with no over/underflow OK to multiply, divide, add positive numbers Proof: + δ factors can be factored out x i ± x j, where x i and x j are initial data (so exact) (x i + y j )(x i y j )x i+ /(x i y j ) - OK Cancellation when subtracting approximate results dangerous:.345xxx -.345yyy.00000zzz We will compute everything using only allowable expressions

6 Classical Example: A Linear System Solve V y = b, where V is : x... x n x... x n x 3... x n 3... x n... x n n y y y 3. y n = + +. and 0 < x <... < x n. Equivalent to interpolation The Björck-Pereyra method solves V y = b In O(n ) time With small forward error: y i ŷ i O(ɛ) y i With small backward error: If ˆV ŷ = b then V ij ˆV ij O(ɛ) V ij. How does it work?

7 The Björck-Pereyra Method If (x, x, x 3 ) = (,, 3) and b = (,, 4) T then using BP to solve 3 3 y = 4 means y = V b = 4 = Notice: Bidiagonal Decomposition of V (accurate) Checkerboard sign pattern No subtractive cancellation High relative accuracy Questions: Which matrices have bidiagonal decomposition of their inverses? Checkerboard signs? Accurate?

8 The Björck-Pereyra Method Dissected Questions: Which matrices have bidiagonal decomposition of their inverses? Checkerboard signs? Accurate? Answers: All nonsingular matrices do This is Neville elimination in matrix form: = ; = = Checkerboard sign pattern Total positivity (A is TP all minors > 0) Accurate? Yes.

9 ACCURACY OF THE BJÖRCK-PEREYRA METHOD x x x 3 x x x 3 x 3 x 3 x 3 3 x 4 x 4 x 3 4 = x x x x x x 3 x 4 x x 4 x x 3 x x 3 x x 4 x x 4 x x x x x x 3 x x 3 x x 4 x 3 x 4 x 3 Other TP matrices?... Yes TP Cauchy matrices x >... > x n > y >... > y n x y x y x y 3 x y x y x y 3 x 3 y x 3 y x 3 y 3 = (x y ) y y 0 0 x y y y (x y ) y y x y 3 y y (x y ) y y x y 3 y y 3 x y x y x 3 y (x y ) x 3 x x 3 y x 3 x 0 0 (x y ) x x x y x x 0 0 (x y ) x 3 x x 3 y x 3 x Unifying Characteristic?

10 The Connection with Minors Which TP matrices permit accurate bidiagonal decomposition? Each entry is product of quotients of minors det(a(i k + : i +, : k)) i+,i = det(a(i k + : i, : k )) L (k) Specifically: Initial minors Contiguous Include first row and column Initial minors of Cauchy: det(c) = Initial minors of : How did we think of minors? det(a(i k + : i, : k )) det(a(i k + : i, : k)) i<j (x j x i )(y j y i ) i,j (x i + y j ) detv = (x i x j ) Gaussian Elimination and Neville Elimination Each entry of V = LDU is a quotient of minors, so not surprising i>j

11 New results: Generalized Matrices TP Matrices with initial minors that are easy to compute accurately and Generalized V = x... x n x... x n... x n... x n n, G λ = x λ x +λ... x n +λ n x λ x +λ... x n +λ n... x λ n x +λ n... x n +λ n n where x > x > > x n > 0, λ n λ n λ 0, λ = λ λ n Initial Minors for G λ? s λ - called Schur function det(g λ ) = det(v ) s λ (x, x,..., x n ) Polynomial with positive integer coefficients Widely studied in combinatorics [MacDonald], group representation theory Example: det x x 4 x x 4 x 3 x 4 3 = det x x x x x 3 x 3 (x x x 3 +x x +x x +x x 3 +x x 3+x x 3 +x x 3),

12 Accuracy and Efficiency for Generalized Matrices Example: det x x 4 x x 4 x 3 x 4 3 = det x x x x x 3 x 3 (x x x 3 +x x +x x +x x 3 +x x 3+x x 3 +x x 3) Accuracy? det(v ) = i>j(x i x j ) - YES. s λ - polynomials with > 0 coefficients - YES. Efficiency? det(v ) = i>j(x i x j ) - OK. s λ (x, x,..., x n )? Traditional algorithm - exponential n λ Now exponential speedup: Linear complexity in n. Idea: s () (x,..., x n ) = i j cost: 3n, although n terms. x i x j = (x +...+x n )x +(x +...+x n )x +...+(x n +x n )x n +x n x n

13 Type of Any GENP Ax = b Ax = b Matrix det(a) A minor GEPP GECP SVD NENP Frwrd* Bckwrd* Cauchy n n n n 3 n 3 n 3 n n TP Cauchy n n n n 3 n 3 n 3 n n n n n 3 n n TP n n 3 EXP n 3 EXP n 3 n n n Polynomial n n 3 n 3 Orth. Poly. Poly. Vand. Orth. poly. ) n n 3 EXP n 3 EXP n 3 0 < x <... < x n Generalized TP Generalized Λn + n Λn + n 3 EXP Λn EXP EXP Λn Λn Λn Big-O sense *FORWARD BOUND: x ˆx O(ɛ) A b, implying x ˆx O(ɛ) x for x checkerboard BACKWARD BOUND: A  O(ɛ) A, where ˆx = b. ) + Other conditions on the signs of the three-term recurrence Λ (λ + )(λ + )... (λ p + ) p, where λ = (λ,..., λ p ).

14 Conclusions TP Structured linear systems can be solved very accurately, if initial minors factor Implies accurate A New application: Generalized Matrices Accurate SVD of some Polynomial Matrices Sometimes the SVD is easier than the inverse Open Problems Totally Positive Matrices in general appear impossible. Proof? Characterize which structured matrices permit accurate and efficient linear algebra

15 Resources These slides: Reports: J. Demmel and P. Koev, Necessary and sufficient conditions for accurate and efficient rational function evaluation and factorizations of rational matrices. In Structured matrices in mathematics, computer science, and engineering. II (Boulder, CO, 999), pages Amer. Math. Soc., Providence, RI, 00. J. Demmel and P. Koev, Accurate and Efficient Matrix Computations with Totally Positive Generalized Matrices Using Schur Functions, J. Demmel, M. Gu, S. Eisenstat, I. Slapničar, K. Veselić, and Z. Drmač. Computing the singular value decomposition with high relative accuracy. Lin. Alg. Appl., 99( 3): 80, J. Demmel, Accurate SVDs for Structured Matrices, Accurate SVDs of structured matrices. SIAM J. Mat. Anal. Appl., ():56 580,