in Numerical Linear Algebra

Transcription

1 Exploiting ResearchTropical MattersAlgebra in Numerical Linear Algebra February 25, 2009 Nick Françoise Higham Tisseur Director Schoolof ofresearch Mathematics The School University of Mathematics of Manchester Joint work with J. Hook (Bath), J. Pestana (Strathclyde), M. Van Barel (Leuven), L. Grammont, J. Hogg, V. Noferini, J. Scott, M. Sharify. 27th Biennial Conference on NA, / 6

2 What Is Tropical Algebra? By tropical" we refer to a semiring in which the addition operation is min or max. In this talk, we consider the max-plus semiring R max = (R,, ), where R = R { }, a b = max(a, b), a b = a + b, a, b R, and additive and multiplicative identities and 0: a = a, a 0 = a. (James Hook used the min-plus semiring in his talk.) Françoise Tisseur Tropical algebra 2 / 30

3 What Is Tropical Algebra? By tropical" we refer to a semiring in which the addition operation is min or max. In this talk, we consider the max-plus semiring R max = (R,, ), where R = R { }, a b = max(a, b), a b = a + b, a, b R, and additive and multiplicative identities and 0: a = a, a 0 = a. (James Hook used the min-plus semiring in his talk.) Tropical algebra is the tropical analogue of linear algebra, working with matrices with entries in R. If A, B R n n, n (A B) ij = a ij b ij, (A B) ij = a ik b kj. k=1 Françoise Tisseur Tropical algebra 2 / 30

4 Valuation A valuation is a map from a field F R (provide a measure of of the size or multiplicity of elements of F). x C V c (x) = log x R max (log 0 = ). For A C n n, V c (A) = (log a ij ). For x, y C, V c (xy) = V c (x) + V c (y), and when x y or x y then V c (x + y) max{v c (x), V c (y)}. Françoise Tisseur Tropical algebra 3 / 30

5 How Can Tropical Algebra Help NLA? Some NLA problems are easier to study/solve when expressed in the tropical algebra setting, e.g., characterization of all Hungarian scalings of a matrix A. Tropical analogues of NLA problems offer order of magnitude approximation, e.g., for entries in the LU factors or Cholesky factors of a matrix A, roots of scalar polynomials and eigenvalues of matrices/matrix polynomials. These approximations are often cheap to compute and useful for the design of preprocessing steps. Françoise Tisseur Tropical algebra 4 / 30

6 Hungarian Scaling A two-sided diagonal scaling of A C n n applied along with permutation P to Ax = b: H = PD 1 AD 2, Hy = PD 1 b, x = D 2 y, with D 1, D 2 diagonal and H = (h ij ) s.t. h ij 1, h ii = 1. Françoise Tisseur Tropical algebra 5 / 30

7 Hungarian Scaling A two-sided diagonal scaling of A C n n applied along with permutation P to Ax = b: H = PD 1 AD 2, Hy = PD 1 b, x = D 2 y, with D 1, D 2 diagonal and H = (h ij ) s.t. h ij 1, h ii = 1. Improves stability of LU fact. with no pivoting [Olschowka & Neumaier 96], [Hogg & Scott 14] Effective preprocessing step for preconditioned iterative methods. [Benzi, Haws & Tůma 00] Code MC64 in HSL. [Duff & Koster 01] Worst case complexity: O(nτ + n 2 log(n)), τ = nnz(a). In practice, complexity is O(τ). Françoise Tisseur Tropical algebra 5 / 30

8 Hungarian Pairs Max-plus permanent of A R n n : perm(a) = π Π n n j=1 a π(j)j = max π Π n n a π(j)j, (1) j=1 Π n : set of permutations of {1,..., n}. Permutation π attaining max in (1) is an optimal assignment. Françoise Tisseur Tropical algebra 6 / 30

9 Hungarian Pairs Max-plus permanent of A R n n : perm(a) = n a π(j)j = max π Π n π Π n j=1 n a π(j)j, (1) Π n : set of permutations of {1,..., n}. Permutation π attaining max in (1) is an optimal assignment. Express (1) as a linear programming problem (LPP) perm(a) = max { n n n a ij d ij : d ij > 0, d ij = d ji = 1 i } i,j=1 with dual problem { n } perm(a) = min u i +v i : u, v R n, a ij u i v j 0. (2) i=1 j=1 A Hungarian pair is a solution (u, v) to (2). j=1 j=1 Françoise Tisseur Tropical algebra 6 / 30

10 Set of All Hungarian Pairs Let π and (u, v) be an opt assignment and a Hungarian pair of A = V c (A) with A C n n. Then H = P π diag 0 ( exp( u) ) Adiag0 ( exp( v) ) C n n is a Hungarian scaling of A. Theorem (Hook, Pestana, T., Hogg 17) The set of all Hungarian pairs of A is given by Hung(A) = { (u + s π 1, v s) : s col(h ) R n}, where H = P π diag ( u) A diag ( v) and (s π 1) i = s π 1 (i). Here H = I H H 2 H n 1 is the Kleene star and col(h) := {H x : x R n } is the column space of H. Françoise Tisseur Tropical algebra 7 / 30

11 Summary/Comments We have used max-plus algebra to characterize set of all Hungarian scalings for a given A C n n, shown that max-balancing a Hungarian scaled matrix yields the most diagonally dominant" Hungarian scaled matrix with respect to some ordering. For max-balanced Hungarian scaled matrices, numerical experiments show reduced need for row interchanges in GEPP, improved stability of LU with no pivoting, improved convergence rate of iterative methods. (See Hook, Pestana, Tisseur, Hogg, MIMS Eprint ) Françoise Tisseur Tropical algebra 8 / 30

12 Objectives Given A C n n, sparse with nonzero entries that vary widely in magnitude, approximate efficiently the order of magnitude of the entries in the LU factors of A, use large entries to define pattern matrix for an ILU preconditioner. Françoise Tisseur Tropical algebra 9 / 30

13 Objectives Given A C n n, sparse with nonzero entries that vary widely in magnitude, approximate efficiently the order of magnitude of the entries in the LU factors of A, use large entries to define pattern matrix for an ILU preconditioner. Use max-plus algebraic techniques: transform A into a max-plus matrix using valuation V c : C R := R { }, V c (x) = log x, (log 0 = ). Françoise Tisseur Tropical algebra 9 / 30

14 Basis for Approximation Entries in L and U can be expressed explicitly in terms of determinants of submatrices of A C n n, e.g., l ik = det ( A([1 : k 1, i], 1 : k) ) / det ( A(1 : k, 1 : k) ), i k, when A has large variation in the size of its entries, V c ( det(a) ) perm ( Vc (A) ), (Heuristic 1) V c : C R, V c (x) = log x, (log 0 = ), perm is the max-plus permanent, i.e., for A R n n, perm(a) = max π Π(n) n a i,π(i). i=1 Françoise Tisseur Tropical algebra 10 / 30

15 Example Let V c (x) = log 10 x and consider [ 10 0 ] 1000 A = , det(a) = Then A = V c (A) = [ 1 3 ] 0 1, 0 0 perm(a) = max{ , } = 3, which provides an approximation of log 10 det(a) Françoise Tisseur Tropical algebra 11 / 30

16 Max-Plus LU Factors of A R n n Let L = (l ij ) and U = (u ij ) R n n be such that l ik = u kj = if i, j < k, and for i, j k, l ik = perm ( A([1:k 1, i], 1: k) ) perm ( A(1: k, 1: k) ), u kj =perm ( A(1:k, [1:k 1, j]) perm ( A(1: k 1, 1: k 1) ). As a consequence of Heuristic 1 we have Heuristic 2: If A = V(A) R n n has max-plus LU factors L, U R n n then A C n n has LU fact A = LU with V c (L) L, V c (U) U. Françoise Tisseur Tropical algebra 12 / 30

17 Example (Cont.) The matrix A = and A = LU = [ ] [ A = V c (A) = has max-plus LU factors [ 0 ] L = 1 0, U = 1 0 Note that V c (L) = L and V c (U) U. has LU fact ] [ 10 0 ] [ 1 3 ] [ 1 ] Françoise Tisseur Tropical algebra 13 / 30

18 Quality of Max-plus LU Approximation 233 matrices from U. Florida sparse matrix collection. precision = (# of true positives)/(# of l ij, u ij 10 2 ), P(p) = % test matrices with precision p, SP(p) = % of test matrices with soft precision p. log 10 l ij, log 10 u ij log 10 l ij, log 10 u ij false -ve true -ve true +ve -1 soft true +ve l ij, u -2 ij l ij, u -2 ij soft false +ve true -ve -3 p P(p) 86% 83% 80% SP(p) 93% 91% 89% Françoise Tisseur Tropical algebra 14 / 30

19 Computing the Max-plus LU Factors Let A R n n have max-plus LU factors L, U given by l ik =perm ( A([1:k 1, i], 1: k) ) perm ( A(1: k, 1: k) ), u kj =perm ( A(1:k, [1:k 1, j]) perm ( A(1: k 1, 1: k 1) ). Françoise Tisseur Tropical algebra 15 / 30

20 Computing the Max-plus LU Factors Let A R n n have max-plus LU factors L, U given by l ik =perm ( A([1:k 1, i], 1: k) ) perm ( A(1: k, 1: k) ), u kj =perm ( A(1:k, [1:k 1, j]) perm ( A(1: k 1, 1: k 1) ). Proposition (Hook, T. 16) Let G = (X, Y ; E) be bipartite graph of A and M l be max weighted matching between {x(i)} l i=1 and {y(i)}l i=1. u kj is the weight of the maximally weighted path through the residual graph R G (M k 1 ) from x(k) to y(j) for j k, or if there is no such a path, l ik is the weight of the maximally weighted path through R T G (M k) from x(k) to x(i) for i > k, or if there is no such a path. Françoise Tisseur Tropical algebra 15 / 30

21 Max-plus ILU Preconditioner Compute Hungarian scaling H = PD 1 AD 2 of A C n n. Compute max-plus LU factors L and U of V c (H). For a threshold t, define pattern matrix as { 1 if lij log t or u S ij = ij log t, 0 otherwise. Compute ILU factors for H restricted to patter matrix S using, for example, the general static pattern ILU alg. Françoise Tisseur Tropical algebra 16 / 30

22 Performance Profile 233 matrices from U. of Florida sparse matrix collection. Cost measure: # iters ( (nnz(h) + nnz(l ilu ) +nnz(u ilu ) ). Tolerance for GMRES: 10 5 (no restart). 1 Cost of GMRES solve (maxit = 100, right precond) 0.8 within α of best threshold ILU (15 fails) max-plus ILU (38 fails) ILU(k) (59 fails) ILU(0) (88 fails) α Françoise Tisseur Tropical algebra 17 / 30

23 Summary/Comments We presented a new method for approximating order of magnitude of entries in LU factors of a A C n n, which uses max-plus algebra and is based solely on a ij. Cost: O ( nτ + n 2 log n ). Can be parallelized. Approximation can be used to compute an ILU preconditioner for A. Max-plus ILU preconditioner tends to outperform ILU(k) and have performance very close to threshold ILU. (see Hook, Tisseur, MIMS Eprint ) Can also define max-plus Cholesky factors and design incomplete Cholesky factorization preconditioners. (see Hogg, Hook, Scott, Tisseur, MIMS Eprint ) Françoise Tisseur Tropical algebra 18 / 30

24 Polynomial Eigenvalue Problem (PEP) Find λ C { } (eigenvalue) and nonzero x, y C n (right/left eigenvectors) s.t. where P(λ) = d j=0 λj P j. P(λ)x = 0, y P(λ) = 0. Assume P(λ) regular, i.e., det P(λ) 0. P has dn eigenvalues. Finite eigenvalues are roots of det P(λ) = 0. PEPs commonly solved by linearization: converts P into a nd nd linear pencil A λb, solve generalized eigenvalue problem (A λb)z = 0, w (A λb) = 0, recover e vecs x, y of P(λ) from those of A λb. Françoise Tisseur Tropical algebra 19 / 30

25 Tropical Scalar Polynomials (Max-Plus) p(z) := d j=0 p j z j = max 0 j d (p j + jz), p j R max. p(z) is a convex, piecewise-affine function. Max-plus roots are the points at which p(z) is non-differentiable, i.e., points at which the maximum expression for p(z) is attained by more than one term. p(z) has d max-plus roots counting multiplicities, α j, j = 1,..., d. p(z) = p d (α 1 z) (α d z). Françoise Tisseur Tropical algebra 20 / 30

26 Tropical Scalar Polynomials (Max-Plus) p(z) := d j=0 p j z j = max 0 j d (p j + jz), p j R. Max-plus roots can be obtained via Newton polygons (upper convex hull of points (j, p j ), j = 0: d). p j p kj p kj 1 Tropical roots: α j = p k j p kj 1 k j k j 1, j = 1,..., q, multiplicity: m j = k j k j 1. p k2 p k1 p k0 p k q k 0 =0 k 1 k 2 k j 1 k j k q 1 k q=d j Françoise Tisseur Tropical algebra 21 / 30

27 Scalar Polynomials: Classical/Max-Plus Tropicalize" p(x) = d i=0 a ix i, a i C, i.e., construct p(z) = d i=0 log a i z i = max 0 i d (log a i + iz). Let α 1 < < α q be roots of p with α j of multiplicity m j. Theorem (Sharify 11) If max(α j α j 1, α j+1 α j ) log for 1 j q then p(x) has exactly m j roots in the annulus A(x) = {x C : 1 3 exp(α j) x 3 exp(α j ) }. Max-plus roots of p(z) offer order of magnitude approx. to roots of p as long as the α j are well separated. Françoise Tisseur Tropical algebra 22 / 30

28 Computation and Applications Max-plus roots of max-plus polynomials can be computed in O(d) operations, where d = deg(p), provide asymptotic growth rates of roots of p(x; t) = d j=0 x j α j (t); have been used for many years in MPSolve (Multiprecision Polynomial Solver) for the selection of the starting points in the Ehrlich-Aberth method. [Bini and Fiorentino, 2000] Françoise Tisseur Tropical algebra 23 / 30

29 Extension to Matrix Polynomials Let P(λ) = d P i λ i C[λ] n n and p(x) = i=0 d log P i x i with max-plus roots α 1 < < α q, α j of multiplicity m j. k 0 < < k q : corresponding indices in Newton polygon. Theorem (Noferini, Sharify, T. 14) If α l α l 1 2 log(1 + 2κ ( P kl ) ), l = j 1, j then P(λ) has exactly nm j ei vals inside the annulus A ( (1 + 2κ(P kj 1 )) 1 exp(α j ), (1 + 2κ(P kj ))exp(α j ) ). i=0 For P kj 1, P kj well conditioned and α j 1, α j, α j+1 sufficiently well separated, P has nm j ei vals of modulus close to exp(α j ). Here κ(a) = A A 1. Françoise Tisseur Tropical algebra 24 / 30

30 Example: Random Cubic n = 10; A0 = randn(n); A1 = 1e3*randn(n); A2 = 1e2*randn(n); A3 = 1e-2*randn(n); λ j α j Eigenvalue index j Françoise Tisseur Tropical algebra 25 / 30

31 Use of Max-Plus Roots in NLA Max-plus roots used to select starting points in the Ehrlich-Aberth method for polynomial eigenproblems. [Bini, Noferini, Sharify 13] Define eigenvalue parameter scalings (λ = exp(α j )µ) for polynomial eigensolvers based on linearizations. P(µ) := δ 1 P(exp(α j )µ), δ = P kj 1 exp(k j 1 α j ). Allow computation of ei pairs with small b err for λ near exp(α j ). Linearization process does not affect ei val condition number of ei vals near exp(α j ). Available in quadratic eigensolver quadeig. [Hammarling, Munroe, Tisseur 13] Françoise Tisseur Tropical algebra 26 / 30

32 Lagrange Linearization Rewrite n n P(λ) = λ 2 M + λd + K in Lagrange basis, P(λ) = l(λ)m + β 1 l 1 (λ)p(σ 1 ) + β 2 l 1 (λ)p(σ 2 ), β j = (σ j σ i ), l j (λ) = λ σ i, i j, l(λ) = l 1 (λ)l 2 (λ). Construct 3n 3n pencil A λb, where M β 1 P(σ 1 )/σ 1 β 2 P(σ 2 )/σ A = I n I n 0, B = 0 1 σ 1 I n 0. 1 I n 0 I n 0 0 σ 2 I n L(λ) is a linearization of λ 3 0 n + P(λ). Use tropical roots for interpolation points σ j, j = 1, 2. Can show that β j P(σ j ) / σ j = O(1), i.e., A is well-balanced. Françoise Tisseur Tropical algebra 27 / 30

33 Numerical Experiments Backward error for eigenpair (λ, x) of P(λ) = d j=0 λj P j, η(λ, x) = P(λ)x 2 (. d j=0 λ j P j 2 ) x 2 Compare Alg.1: QZ alg applied to tropically scaled Lagrange linearization. [Van Barel, Tisseur 17] Alg.2: MATLAB s polyeig function. Alg.3: Gaubert & Sharify s algorithm. [Gaubert, Sharify 09] Françoise Tisseur Tropical algebra 28 / 30

34 Numerical Experiments (Cont.) Largest backward errors of eigenpairs computed by Alg.1 Alg.3. Problem d n Alg.1 Alg.2 Alg.3 cd_player e e e-13 damped_beam e e e-17 hospital e e e-15 orr_sommerfeld e e e-15 power_plant e e e-18 Problem e e e-12 Problem e e e-14 Problem e e e-10 Françoise Tisseur Tropical algebra 29 / 30

35 Conclusion Some NLA problems are easier to solve in the tropical algebra setting. Tropical analogues of NLA problems offer approximation to solutions of classical problems. These solutions are usually cheap to compute, usuful for the design of preprocessing steps and scalings. Papers and tech reports available on my web page. Françoise Tisseur Tropical algebra 30 / 30