Structured Matrices and Solving Multivariate Polynomial Equations

Size: px

Start display at page:

Download "Structured Matrices and Solving Multivariate Polynomial Equations"

Crystal Stevens
5 years ago
Views:

1 Structured Matrices and Solving Multivariate Polynomial Equations Philippe Dreesen Kim Batselier Bart De Moor KU Leuven ESAT/SCD, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium. Structured Matrix Days, XLIM, Université de Limoges. May 10-11, 2012, Limoges, France.

2 Outline 1 Introduction & Motivation 2 Solving Polynomial Systems 3 Numerical Example 4 Conclusions & Future Work

3 Outline 1 Introduction & Motivation 2 Solving Polynomial Systems 3 Numerical Example 4 Conclusions & Future Work

4 Univariate Root-finding: Sylvester Consider two univariate polynomials { f1(x) = a rx r +a r 1x r a 0 f 2(x) = b sx s +b s 1x s b 0 Build square Sylvester matrix a 0 a 1... a r s rows a 0 a 1... a r r rows or, a 0 a 1... a r b 0 b 1... b s b 0 b 1... b s b 0 b 1... b s Sk = 0 x 0 x 1 x 2. x r+s 2 x r+s 1 = /25

5 Univariate Root-finding: Sylvester Outline univariate root-finding algorithm 1) Build Sylvester matrix Sk = 0 2) Shift structure in k: ( 1 x x 2... x r+s 2 ) T x = ( x x 2 x 3... x r+s 1 ) T is written as kx = k where and represent omitting the last and the first row 3) Shift structure for all roots: KD x = K 4) One cannot compute K directly, but instead Z = KV, such that VD xv 1 = Z Z with X denoting the Moore-Penrose pseudoinverse of X. 2/25

6 Univariate Root-finding: Sylvester Example: consider the system { f1 (x) = (x 1)(x 2) = x 2 3x+2 f 2 (x) = (x 1)(x 2)(x+1) = x 3 2x 2 x+2 having two common roots x = 1 and x = 2. The Sylvester matrix S for degree d = = 4 is S = 1 x x 2 x 3 x 4 f xf x 2 f f xf /25

7 Univariate Root-finding: Sylvester The nullspace of S has a dimension of two and a numerical basis is computed using SVD Z = The eigenvalue decomposition of Z Z is = The common roots of f 1 (x) and f 2 (x) are x = 1 and x = 2. 4/25

8 Univariate Root-finding: Sylvester Huge gap between algebraic geometry literature and (numerical) linear algebra Linear algebra provides suitable framework Polynomial structure induces matrix structure Main ingredients: 1) Linearize problem by separating coefficients and monomials 2) Solutions live in the nullspace of coefficient matrix 3) Exploit structure in monomial basis 4) Eigenvalue problems Generalize to multivariate case? 5/25

9 Outline 1 Introduction & Motivation 2 Solving Polynomial Systems 3 Numerical Example 4 Conclusions & Future Work

10 Outline Algorithm and Building Macaulay Matrix M Algorithm ingredients 1) Build structured matrix M containing coefficients 2) Solutions compose vectors in nullspace M 3) Compute solutions from eigenvalue problems Example: simple polynomial system { p(x,y) = x 2 +3y 2 15 = 0 (d 1 = 2) q(x,y) = y 3x 3 2x 2 +13x 2 = 0 (d 2 = 3) Construct Macaulay matrix 1 x y x 2 xy y 2 x 3 x 2 y xy 2 y 3 p(x, y) q(x, y) x p(x,y) y p(x,y) /25

11 Outline Algorithm and Building Macaulay Matrix M Simple polynomial system { p(x,y) = x 2 +3y 2 15 = 0 q(x,y) = y 3x 3 2x 2 +13x 2 = 0 Extend Macaulay matrix M to degree d = n i=1di n+1 = 4 1 x y x 2 xy y 2 x 3 x 2 y xy 2 y 3 x 4 x 3 y x 2 y 2 xy 3 y 4 p q xp yp x 2 p xyp y 2 q xq yq Dimensions of M are determined by number of monomials N(n, d) N(n,d ) = (n+d 1)! (n 1)! d! 7/25

12 Solutions Live in Nullspace of M Solutions compose vectors in nullspace of the Macaulay matrix M where M k = 0, k = ( 1 x y x 2 xy y 2 )T is evaluated at the solutions Number of solutions follows from nullity M Stacking vectors k forms canonical nullspace K with generalized Vandermonde structure K = ( k (1) k (2) k (m B) ) Canonical nullspace K for 6 solutions (x i,y i ) x 1 x 2... x 6 y 1 y... y 6 x 2 1 x x 2 6 x 1 y 1 x 2 y 2... x 6 y 6 y1 2 y y6 2 x 3 1 x x 3 6 x 2 1 y 1 x 2 2 y 2... x 2 6 y 6 x 1 y1 2 x 2 y x 6 y6 2 y1 3 y y6 3 x 4 1 x x 4 6 x 3 1 y 1 x 3 y 2... x 3 6 y 6 x 2 1 y2 1 x 2 2 y x 2 6 y2 6 x 1 y 1 3 x 2 y x 6 y 6 3 y y y /25

13 Shift structure Shift structure in monomial basis k ( ) x = ( ) 1 x y x 2 xy y 2 1 x y x 2 xy y 2 y = ( ( Let D = diag(x 1,x 2,...,x mb ) (for finding the x-roots), then S 1 KD = S 2 K, where S 1 and S 2 select rows from K wrt. shift structure ) ) 1 x y x 2 xy y 2 1 x y x 2 xy y 2 9/25

14 Two Root-finding Algorithms Two flavours of root-finding algorithms 1) Nullspace-based root-finding: compute numerical basis and exploit shift structure 2) Data-driven root-finding: perform (Q)R-decomposition of certain columns of M Both algorithms lead to (generalized) eigenvalue problems! 10/25

15 Two Root-finding Algorithms Nullspace-based Root-finding We have S 1KD = S 2K However, the canonical nullspace K and the solutions D are not known. A basis Z (with MZ = 0) is computed, with K = ZV This leads to the generalized eigenvalue problem S 1 ZVD = S 2 ZV or S 1 Z ( VDV 1) = S 2 Z 11/25

16 Two Root-finding Algorithms Algorithm summary nullspace-based root-finding 1) Construct Macaulay matrix M for degree d 2) Compute basis for nullspace of M as Z 3) Choose shift function, e.g., x 4) Write down shift relation using row selection matrices S 1 and S 2 5) Solve the generalized eigenvalue problem S 1 Z ( VDV 1) = S 2 Z The eigenvalues correspond to the, e.g., x-solutions 6) Reconstruct canonical kernel K = ZV 12/25

17 Two Root-finding Algorithms Data-driven Root-finding Perform partitioning of M such that or, ( M1 M 2 ) ( K 1 K 2 ) = 0, M 1 K 1 +M 2 K 2 = 0, where M 2 is of full column rank, and K 1 is square and of full rank. This leads to K 2 = M 2 M 1K 1, where X denotes the Moore-Penrose pseudoinverse of X. 13/25

18 Two Root-finding Algorithms The shift relation S 1 KD σ = S 2 K is written as K 1 D σ = = ( ) Σ1 K 1 ( Σ 2 K 2 Σ 1 Σ 2 M 2 M 1 ) K 1...necessary to compute entire M 2 M 1?! 14/25

19 Two Root-finding Algorithms We have ( K 1 D σ = Identify two kinds of shifts: Σ 1 Σ 2 M 2 M 1 ) K 1 1) Trivial shifts map mons from K 1 to other mons in K 1 2) Nontrivial shifts map mons from K 1 to K 2 = M 2 M 1K 1 We are interested only in a certain part of M 2 M 1! 15/25

20 Two Root-finding Algorithms Reorder once again the columns of M in the expression MK = 0 ( ) Σ 2 K 2 M22 M 21 M 1 Σ 2 K 2 = 0 K 1 (Q)R helps us to solve for the rows of interest Σ 2 K 2 : R 11 R 12 R 13 R 22 R 23 R 33 Σ 2 K 2 Σ 2 K 2 K 1 = 0 The root-finding problem is phrased as the eigenvalue problem ( ) Σ K 1 D σ = 1 R22 1 R K /25

21 Two Root-finding Algorithms Algorithm summary data-driven root-finding 1) Construct Macaulay matrix M for degree d 2) Partition M = ( M 1 M 2 ) and K = ( K T 1 K T 2 3) Choose shift function σ, e.g., σ = x 4) Identify trivial and nontrivial shifts 5) Perform (Q)R on reordered M to obtain rows of interest R 11 R 12 R 13 R 22 R 23 Σ 2K 2 Σ 2 K 2 = 0 R 33 K 1 6) Solve eigenvalue problem ( K 1 D σ = Σ 1 R 1 22 R 23 ) K 1 ) T 17/25

22 Sparse Nullspace Computation: Motzkin elimination Exploiting sparsity in M based on Motzkin elimination [7] Main idea: pairwise eliminations with nonzero elements Consider a sparse matrix ( ) M = Process the first row: a 1 W 1 = ( ) /25

23 Sparse Nullspace Computation: Motzkin elimination Process the second row: leading to b 2 = a 2 W 1 = ( ) b 2 W 2 = ( ) Finally the full nullspace is found as Z = m i=1 W i 19/25

24 Outline 1 Introduction & Motivation 2 Solving Polynomial Systems 3 Numerical Example 4 Conclusions & Future Work

25 TLS vs W/STLS Total Least Squares [10, 4] Weighted/Structured TLS [1, 2] minimize B,v subject to Bv = 0 (a ij b ij ) 2 i j v T v = 1 Singlar Value Decomposition A = UΣV T Av = uσ A T u = vσ v T v = 1 u T u = 1 Eigenvalue problem! minimize B,v subject to Bv = 0 (a ij b ij ) 2 w ij i j (B structured) v T v = 1 Riemannian Singlar Value Decomposition Av = T vt T v l A T l = T l T T l v v T v = 1 System of polynomial equations! 20/25

26 Applications W/STLS Applications W/STLS Systems and control: [1], [2], [6] Machine learning: [8] Information retrieval: [5] Statistics: [3]... e ũ u 0 y 0 System ỹ w = u 0 +ũ z = y 0 +ỹ 21/25

27 STLS Problem 3 3 Hankel STLS minimize v subject to v T v = 1. v T A T (T v T T v ) 1 Av A = ( ) B = ( ) 22/25

28 Outline 1 Introduction & Motivation 2 Solving Polynomial Systems 3 Numerical Example 4 Conclusions & Future Work

29 Many Applications Polynomial root-finding arises in Polynomial Optimization Problems Model order reduction Identifiability analysis of nonlinear models Signal Processing Kinematic problems in robotics Computational Biology: conformation of molecules Algebraic Statistics... 23/25

30 Summary and Future Work Translation algebraic geometry problem to (numerical) linear algebra Two kinds of root-finding algorithms Polynomial system solving lead to eigenvalue decompositions as in [9] Exploiting sparsity and structure, e.g., FFT-like computations, recursive orthogonalization, etc. Numerical stability and conditioning 24/25

31 Thank You Thank you for your attention! Questions? 25/25

32 References I B. De Moor. Structured Total Least Squares and L 2 approximation problems. Lin. Alg. Appl., : , B. De Moor. Total Least Squares for affinely structured matrices and the noisy realization problem. IEEE Trans. Signal Process., 42(11): , K. R. Gabriel and S. Zamir. Lower rank approximation of matrices by Least Squares with any choice of weights. Technometrics, 21, No. 4, November G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, MD, USA, third edition, E. P. Jiang and M. W. Berry. Information filtering using the Riemannian SVD (R-SVD). In Proc. 5th Int. Symp. Solving Irregul. Struct. Probl. Parallel, pages , I. Markovsky. Structured low-rank approximation and its applications. Automatica, 44: , 2008.

33 References II T. S. Motzkin, H. Raiffa, G. L. Thompson, and R. M. Thrall. The double description method. In Contributions to the theory of games, vol. 2, Annals of Mathematics Studies, no. 28, pages Princeton University Press, Princeton, N. J., N. Srebro and T. Jaakkola. Weighted low-rank approximations. Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), Wachington DC, H. J. Stetter. Numerical Polynomial Algebra. SIAM, S. Van Huffel and J. Vandewalle. The Total Least Squares Problem: Computational Aspects and Analysis, volume 9 of Frontiers in Applied Mathematics. SIAM, Philadelphia, 1991.

Solving Systems of Polynomial Equations: Algebraic Geometry, Linear Algebra, and Tensors?

Solving Systems of Polynomial Equations: Algebraic Geometry, Linear Algebra, and Tensors? Philippe Dreesen Bart De Moor Katholieke Universiteit Leuven ESAT/SCD Workshop on Tensor Decompositions and Applications