Canonical forms of structured matrices and pencils

Canonical forms of structured matrices and pencils Christian Mehl and Hongguo Xu Abstract This chapter provides a survey on the development of canonical forms for matrices and matrix pencils with symmetry structures and on their impact in the investigation of application problems. The survey mainly focuses on the results from three topics that have been developed during the past 15 years: structured canonical forms for Hamiltonian and related matrices structured canonical forms for doubly structured matrices and pencils and singular value-like decompositions for matrices associated with two sesquilinear forms. 1 Introduction Eigenvalue problems frequently arise in several applications from natural sciences and industry and therefore the corresponding theory is a fundamental topic in Linear Algebra Matrix Theory and Numerical Analysis. The practical applications typically lead to matrices matrix pencils or matrix polynomials with additional symmetry structures that reflect symmetries in the underlying physics. As a consequence also the eigenstructures (i.e. eigenvalues eigenvectors root vectors Jordan blocks singular blocks and other invariants as e.g. algebraic geometric and partial multiplicities) of such matrices matrix pencils and matrix polynomials inherit certain symmetries or patterns. As these reflect the nature and characteristics of the original application problems they play critical roles both in theory and practice. Christian Mehl Institut für Mathematik MA 4-5 TU Berlin Str. des 17. Juni 136 1623 Berlin Germanye-mail: mehl@math.tu-berlin.de Hongguo Xu Department of Mathematics University of Kansas Lawrence KS 6645 USA. e-mail: xu@math.ku.edu. Partially supported by Alexander von Humboldt Foundation and by Deutsche Forschungsgemeinschaft through the DFG Research Center MATHEON Mathematics for Key Technologies in Berlin. 1

2 Christian Mehl and Hongguo Xu Typically the solution of structured eigenvalue problems is a challenge because there is demand for the design of new algorithms that are structure-preserving in each step so that the corresponding symmetry in the spectrum is maintained in finite precision arithmetic and the obtained results are physically meaningful 48. Simple variations of the QR algorithm or methods based on standard Krylov subspaces may not be sufficient to achieve this goal so that new ideas and concepts need to be developed. This requires a deeper understanding of the corresponding eigenstructures and therefore the derivation of structured canonical forms is essential. It is the aim of this chapter to review such forms for some particular classes of structured matrices or matrix pencils. The most important and well-known matrices with symmetry structures are probably real or complex Hermitian skew-hermitian and unitary matrices. Still there are many other kinds of important structured matrices like complex symmetric skew-symmetric and orthogonal matrices as well as nonnegative matrices all of which are discussed in the classical books 13 14. In this chapter we focus on structured matrices that are self-adjoint skew-adjoint or unitary with respect to an inner product associated with a possibly indefinite Hermitian or skew-hermitian matrix and give a brief review on their theory also including the corresponding matrix pencils that generalize those structures. We do not consider the corresponding structured matrix polynomials in this chapter but refer the reader to Chapter 12 of this book instead. Let F be either the real field R or the complex field C. Suppose M F m m is an invertible Hermitian or skew-hermitain matrix and define the bilinear or sesquilinear form xy M = x My =: xy xy F m (1) where is the conjugate transpose which reduces to just T the transpose if F = R. Then three sets of structured matrices can be defined: 1) the set of M-Hermitian matrices or M-selfadjoint matrices: H M = { A A M = MA } = { A AxyM = xay M for all xy F m} 2) the set of M-skew-Hermitian matrices or M-skew-adjoint matrices: S M = { K K M = MK } = { K Kxy M = xky M for all xy F m} 3) the set of M-unitary matrices: U M = { U U MU = M } = { U UxUyM = xy M for all xy F m}. The concept of M-Hermitian and M-skew-Hermitian matrices can be generalized to matrix pencils via λm B; with M = ±M B = ±B. (2) In fact if M is invertible the generalized eigenvalue problem with underlying matrix pencil as in (2) is equivalent to the eigenvalue problem for the matrix A = M 1 B which is M-Hermitian or M-skew-Hermitian depending on whether M and B are Hermitian or skew-hermitian. M-unitary matrices may be related to structured pencils indirectly by using a Cayley-transformation 24 28 38.

Canonical forms of structured matrices and pencils 3 Another structured matrix pencil of the form λa A which is called palindromic (22 29 43 44) can also be transformed to a Hermitian/skew-Hermitian pencil with a Cayley-transformation and can therefore be considered a generalization of M-unitary matrices as well. The study of matrices and matrix pencils with the symmetry structures outlined above started about one and a half centuries ago (we refer to the review article 25 and the references therein for more details) and continues to be of strong interest as there are many important applications in several areas of science and engineering see e.g. 16 24 38 24 41 45 48 49 55. A particular example is given by Hamiltonian matrices that arise e.g. in systems and control theory 27 38 45 55 and in the theory of dynamical and Hamiltonian systems 19 2 21. These matrices are J-skew-Hermitian where the skewsymmetric matrix J is given by In J := J n := R 2n 2n. (3) I n (We drop the subscript n whenever it is clear from the context.) Due to their many applications in particular those in system and control theory the investigation of Hamiltonian matrices has been an important part of Volker Mehrmann s research interest and he and his coauthors have contributed many results to their theory like discovering the reason for the difficulty in computing Hamiltonian Hessenberg forms 1 finding necessary and sufficient conditions for the existence of the Hamiltonian Schur form 28 and developing several algorithms for the Hamiltonian eigenvalue problem 4 8 38 39. For the understanding of the underlying theory it was crucial to be aware of the presence of additional invariants besides the eigenvalues eigenvectors and root vectors of Hamiltonian matrices the so called signs in the sign characteristic of purely imaginary eigenvalues. The classical Jordan canonical form cannot display these additional invariants because it is obtained under general similarity transformations that ignore the special structure of Hamiltonian matrices. Therefore it was important to develop a canonical form that is obtained under structure-preserving transformations so that additional information like the sign characteristic is preserved and can be read off. The phenomenon of presence of a sign characteristic not only occurs for the special case of Hamiltonian matrices but for all three types of matrices structured with respect to the inner product (1) induced by M. To be more precise it occurs for real eigenvalues of M-Hermitian purely imaginary eigenvalues of M-skew- Hermitian and unimodular eigenvalues of M-unitary matrices as well as for the classes of related matrix pencils as in (2). In all cases the sign characteristic has proven to play a fundamental role in theory and applications like in the analysis of structured dynamic systems 19 2 21 in perturbation analysis of structured matrices 31 32 33 4 and in the investigation of solutions of Riccati equations 12 24 28 to name a few examples.

4 Christian Mehl and Hongguo Xu After introducing the well-known canonical forms for Hermitian pencils and M- Hermitian matrices in the next section we will give a survey on three related topics in the following sections: (a) structured canonical forms for Hamiltonian and related matrices (b) canonical forms for doubly structured matrices (c) singular value-like decompositions for matrices associated with two sesquilinear forms. Throughout the chapter we will use the following notation. A 1 A m is the block diagonal matrix diag(a 1...A m ). The n n identity matrix is denoted by I n and m n ( n ) stand for the m n (n n) zero matrix. If the size is clear from the context we may use I and instead for convenience. We denote by e j the jth unit vector i.e. the jth column of I. The n n reverse identity will be denoted by R n while J n (α) stands for the upper triangular n n Jordan block with eigenvalue α that is 1 R n =... α 1... F n n...... J n (α) = 1... 1 α Fn n Finally the m (m + 1) singular block in the Kronecker canonical form of matrix pencils is denoted by λ 1 L m (λ) =....... λ 1 2 Canonical forms for Hermitian pencils and M-Hermitian matrices For all the structured pencils of one of the forms in (2) the theory of structured Kronecker canonical forms is well-established see e.g. 9 25 26 46 47 following the work from the second half of the 19th century 23 52 51. These forms are obtained under congruence transformations (λm B) X (λm B)X with X invertible because those preserve both the Hermitian and the skew-hermitian structure of matrices and thus the structure of pencils λm B of the forms in (2). For instance a Hermitian pencil λm B i.e. a pencil such that both M and B are Hermitian has the following structured Kronecker canonical form under congruence. Theorem 1. Let λm B be a complex n n Hermitian pencil. Then there exists an invertible matrix X such that X (λm B)X = J C (λ) J R (λ) J (λ) L (λ) (4)

Canonical forms of structured matrices and pencils 5 where J C (λ) = ( Rm1 λ R m1 ( λ Rmp R mp R m1 J m1 (λ 1 ) R m1 J m1 ( λ 1 ) ) R mp J mp (λ p ) R mp J mp ( λ p ) ) J R (λ) = s 1 R n1 ( λin1 J n1 (α 1 ) ) s q R nq ( λinq J nq (α nq ) ) ( ) ( ) J (λ) = s q+1 R k1 λjk1 () I k1 sq+r R kr λjkr () I kr L L (λ) = l1 (λ) L L l1 (λ) T lt (λ) L lt (λ) T ν ν with Imλ j > j = 1... p; α j R j = 1...q; s j = ±1 j = 1...q + r and pqrt N. If two pencils C(λ) and D(λ) are equivalent i.e. X 1 C(λ)X 2 = D(λ) for some invertible matrices X 1 X 2 independent of λ we use the notation C(λ) D(λ). It is easy to show that the blocks in (4) satisfy J C (λ) ( λi m1 J m1 (λ 1 ) ) ( λi m1 J m1 ( λ 1 ) ) ( λi mp J mp (λ p ) ) ( λi mp J mp ( λ p ) ) J R (λ) ( λi n1 J n1 (α 1 ) ) ( λi nq J nq (α q ) ) J (λ) ( λj k1 () I k1 ) ( λjks () I ks ) L (λ) L l1 (λ) L T l 1 (λ) L lt (λ) L T l t (λ) ν ν. Therefore the classical Kronecker canonical form of the pencil λm B can easily be read off from the structured version (4). In particular the pairing of blocks elegantly displays the corresponding symmetry in the spectrum: the block J C (λ) contains the nonreal eigenvalues that occur in complex conjugate pairs λ j λ j both having exactly the same Jordan structures. If the pencil is singular then the singular blocks - contained in L (λ) - are also paired: each right singular block L l j (λ) has a corresponding left singular block L l j (λ) T. However the structured canonical form (4) has an important advantage over the classical Kronecker canonical form of a Hermitian pencil. It displays additional invariants that are present under concruence transformations the signs s 1...s q+r attached to each Jordan block of a real eigenvalue and each Jordan block of the eigenvalue infinity. The collection of these signs is referred to as the sign characteristic of the Hermitian pencil 25 see also 15. As a corollary of Theorem 1 one obtains a canonical form for M-Hermitian matrices also known as M-selfadjoint matrices see 15 25. Corollary 1. Let M C n n be Hermitian and invertible and let A C n n be M- Hermitian. Then there exists an invertible matrix X C n n such that X 1 AX = J R J C X MX = M R M C

6 Christian Mehl and Hongguo Xu where J R = J n1 (α 1 ) J nq (α q ) Jm1 (λ J C = 1 ) J m1 ( λ 1 ) Jmp (λ p ) J mp ( λ p ) M r = s 1 R n1 s q R nq M C = R 2m1 R 2mp where α j R s j = ±1 j = 1...q Imλ j > j = 1... p and pq N. Indeed the form is easily obtained by recalling that a matrix A is M-Hermitian if and only if the pencil λm MA is a Hermitian pencil and applying Theorem 1 to this pencil. By convention we will call s j in Corollary 1 the sign of the Jordan block J n j (α j ). For the other three types of matrix pencils in (2) structured canonical forms can be derived directly from (4). If λ M B is Hermitian/skew-Hermitian skew- Hermitian/Hermitian or skew-hermitian/skew-hermitian then Theorem 1 can be applied to the Hermitian pencils λm ( ib) λ( im) B or λ( im) ( ib) respectively to obtain (4). As a consequence these pencils also have a sign characteristic. In the case of pencils of mixed structure i.e. one matrix being Hermitian and the other skew-hermitian now the purely imaginary eigenvalues (including the eigenvalue infinity) have signs. For the case of real pencils of the form (2) also real structured Kronecker canonical forms under real congruence transformations are known. We refer the reader to 26 47 for details. 3 Structured canonical forms for Hamiltonian matrices When the matrix defining the inner product (1) is the skew-symmetric matrix J from (3) then a J-Hermitian matrix is called skew-hamiltonian a J-skew-Hermitian is called Hamiltonian and a J-unitary matrix is called symplectic. In many applications in particular in systems and control invariant Lagrangian subspaces are of interest. A Lagrangian subspace is an n-dimensional subspace L F 2n that is J-neutral i.e. xy J = for all xy L. Suppose the columns of the matrix W 1 F 2n n span an invariant Lagrangian subspace L of a Hamiltonian matrix H F n n. Then there exists a 2n n matrix W 2 such that W = W 1 W 2 is sympectic. Indeed one may choose W 2 = J T W 1 (W 1 W 1) 1. Then W W JW = 1 JW 1 W1 JW 2 W2 JW 1 W2 JW 2 = In = J I n because W1 JW 1 = = (W1 W 1) 1 W1 JJJT W 1 (W1 W 1) 1 = W2 JW 2 as L is J-neutral. Since L is also an invariant subspace of H we obtain W 1 T D HW = T D = D. (5)

Canonical forms of structured matrices and pencils 7 From the decomposition (5) we can easily see that a necessary condition for the existence of an invariant Lagrangian subspace is that the algebraic multiplicities of all purely imaginary eigenvalues must be even because any purely imaginary eigenvalue iα of T is also an eigenvalue of T. This condition however is not sufficient as the following example shows. Example 1. Consider the Hamiltonian matrices 1 1 H 1 = 1 1 = J 2 H 2 = 1 1 = J1 J T 1 1 1 Then the matrix H 1 does not have a decomposition (5) since for any symplectic matrix W by definition W H 1 W = W J 2 W = J 2. The matrix H 2 on the other hand already is of the form (5). Surprisingly the matrices H 1 and H 2 are similar. It is easy to check that they both have the semi-simple eigenvalues i and i which both have the algebraic multiplicity two. To explain this surprising behavior a closer look at a structured canonical form of Hamiltonian matrices is necessary. One way is to consider instead of a Hamiltonian matrix H the ij-hermitian matrix ih and to apply Corollary 1. This yields the existence of an invertible matrix X such that X 1 HX = H I H C X JX = M I M C where H I = ij n1 (α 1 ) ij nq (α q ) M I = s 1 ir n1 s q ir nq Jmp (λ p ) i Jm1 (λ H C = i 1 ) J m1 ( λ 1 ) J mp ( λ p ) M C = ir 2m1 ir 2mp. Here s j is the sign of the Jordan block J n j (iα j ) of H for j = 1...q i.e. the purely imaginary eigenvalues come with a sign characteristic. Although this canonical form reveals these additional invariants one cannot tell immediately whether a decomposition as (5) exists. One possible way to proceed is to apply further transformations to transform X JX back to J say by constructing an invertible matrix Y such that H := (XY ) 1 H(XY ) (XY ) J(XY ) = J Then the matrix XY is symplectic because (XY ) J(XY ) = J. Clearly there are many such transformations and then the task is to choose among all these transformations a particular one so that H is as close to a block upper triangular form (5) as possible. In 28 such an optimal canonical form is presented in the sense that the (21) block of H has the lowest possible rank. The result is given in the following theorem.

8 Christian Mehl and Hongguo Xu Theorem 2 (28). Let H C 2n 2n be a Hamiltonian matrix. Then there exists a symplectic matrix W C 2n 2n such that T c T ie D ie T io D io W 1 HW = T ior D ior Tc T ie T io M ior where the blocks have the following properties: (i) T ior T c = J m1 (λ 1 ) J m2 (λ 2 ) J mp (λ p ) where λ 1...λ p C with Reλ 1...Reλ p >. (ii) T ie = J n1 (iα 1 ) J nq (iα q ) D ie = s 1 e n1 e n 1 s q e nq e n q where α 1...α q R and s 1...s q = ±1. Each sub-matrix Jn j (iα j ) s j e n j e n j (J n j (iα j )) corresponds to an even-sized Jordan block J 2n j (iα j ) of H with sign s j. (iii) T io = T (1) io T (r) io D io = D (1) io D(r) io and T ( j) io = J l j (iβ j ) 2 2 e l j J k j (iβ j ) 2 2 e k j iβ j D ( j) 2i io = e l j 2 σ j e k j e l j e k j where β 1...β r R and σ 1...σ r = ±1. For each j = 1...r the sub-matrix T ( j) io D ( j) io (T ( j) io ) corresponds to two odd-sized Jordan blocks of H associated with the same purely imaginary eigenvalue iβ j. The first is J 2l j +1(iβ j ) with sign σ j and the second is J 2k j +1(iβ j ) with sign σ j. (iv) T ior = T (1) (t) ior T ior M ior = M (1) ior M(t) ior D ior = D (1) ior D(t) ior

Canonical forms of structured matrices and pencils 9 where T ( j) ior = J η j (iγ j ) 2 2 e η j J ν j (iδ j ) 2 2 e ν j i 2 (γ j + δ j ) D ( j) 2i ior = 2 σ r+ j M ( j) ior = σ r+ j e η j e ν j e η j e ν j 2i 2 (γ j δ j ) and γ j δ j R γ j δ j σ r+ j = ±1 for j = 1...t. The submatrix T ( j) ior D ( j) ior M ( j) ior (T ( j) ior ) 1 2 (γ j δ j ) corresponds to two odd-sized Jordan blocks of H associated with two distinct purely imaginary eigenvalues iγ j and iδ j. The first one is J 2η j +1(iγ j ) with sign σ r+ j and the second one is J 2ν j +1(iδ j ) with sign σ r+ j. Thus the spectrum of H can be read of from the Hamiltonian submatrices H ie := H io := H ior := Tc H c := Tc Tie D ie T ie Tio D io T io Tior D ior M ior T ior of smaller sizes. The submatrix H c contains all Jordan blocks associated with eigenvalues that are not purely imaginary. To be more precise T c contains all the Jordan blocks of eigenvalues of H with positive real parts and Tc contains all the Jordan blocks of eigenvalues of H with negative real parts. The submatrix H ie contains all even-sized Jordan blocks associated with purely imaginary eigenvalues of H whereas H io and H ior contain all Jordan blocks associated with purely imaginary eigenvalues of H that have odd sizes. Here H io consists of pairs of Jordan blocks of (possibly different) odd sizes that are associated with the same purely imaginary eigenvalue but have opposite signs. On the other hand H ior consists of the remaining Jordan blocks that do not allow such a pairing. In particular if H ior contains more than one Jordan block associated to a particular purely imaginary eigenvalue then all such blocks must have the same sign in the sign characteristic. While the canonical form in Theorem 2 looks quite complicated at first sight its advantage is that the conditions for the existence of a decomposition of the form (5) can now be trivially derived by requesting the submatrix H ior being void. Thus with the interpretation of H ior in terms of the sign characteristic we immediately obtain the following result that is in accordance with a corresponding result in 42 in terms of M-selfadjoint matrices. Theorem 3 (28). A Hamiltonian matrix H has a decomposition (5) if and only if for each purely imaginary eigenvalue of H it has an even number of odd-sized Jordan blocks half of which have sign +1 and half of which have sign 1.

1 Christian Mehl and Hongguo Xu The theorem also gives necessary and sufficient conditions for the existence of the Hamiltonian Schur form. A Hamiltonian matrix H is said to have a Hamiltonian Schur form if it allows a decomposition of the form (5) with T being upper triangular and W being both symplectic and unitary i.e. satisfying W JW = J and W W = I. Under the same conditions as in Theorem 3 we obtain the existence of a symplectic matrix W such that W 1 HW is in the Hamiltonian canonical form of Theorem 2 without the blocks from H ior. Since the blocks T c T ie and T io are upper triangular we find that W 1 HW has the form (5) with T being upper triangular. A Hamiltonian Schur form can then be derived by performing a symplectic QR-like decomposition to the symplectic matrix W see 7 28. Corollary 2 (28). Let H be a Hamiltonian matrix. Then there exists a unitary and symplectic matrix W such that W 1 HW has the form (5) with T being upper triangular if and only if for each purely imaginary eigenvalue H has an even number of odd-sized Jordan blocks half of which have sign +1 and half of which have sign 1. The following example borrowed from 3 shows that the two Jordan blocks that are paired in one of the particular submatrices of H io in Theorem 2 may indeed have different sizes. Example 2. Consider the two matrices i 1 1 2i i 2i H = i i and X = 1 1 i i 2 1. 1 i i 1 1 Then H is a Hamiltonian matrix in Hamiltonian Schur form and X is the transformation matrix that brings the pair (ihij) into the canonical form of Corollary 1: 1 1 1 X 1 (ih)x = 1 1 1 X (ij)x = 1 1 1 1 Thus H has the eigenvalue i and two corresponding Jordan blocks with sizes 3 and 1. The Jordan block of size 3 has the sign +1 and the Jordan block of size 1 has the sign 1 thus satisfying the condition of Theorem 3. Example 3. Revisiting the matrices H 1 and H 2 from Example 1 one can easily check that the eigenvalues i and i of H 2 have one Jordan block with sign +1 and one Jordan block with sign 1 each. In fact H 2 is a matrix of the form H io as in Theorem 2. On the other hand for the matrix H 1 the signs corresponding to i are both +1 and the signs corresponding to i are both 1. In fact H 1 is in the canonical form of Theorem 2 corresponding exactly to a matrix in the form H ior. However for the matrix X = e 1 e 3 e 2 e 4 which is not symplectic we obtain that X 1 H 1 X = H 2 is in the form (5). Although the transformation with X maps the

Canonical forms of structured matrices and pencils 11 Hamiltonian matrix H 1 to the Hamiltonian matrix H 2 it is not a structure-preserving transformation in the sense that for small Hamiltonian perturbations H 1 + H the transformed matrix H 2 + X 1 HX is in general not Hamiltonian. This fact in a sense allows the similarity transformation with X to take a bypass by ignoring the sign constraints shown in Theorem 3. It was shown in 28 that the existence of the decomposition (5) with a non-symplectic similarity transformation only requires the algebraic multiplicities of all purely imaginary eigenvalues of H to be even. In the case that the Hamiltonian matrix under consideration is real there is also a canonical form under real symplectic similarity see 28 Theorem 22. In this case the eigenvalues of a Hamiltonian matrix are not only symmetric with respect to the imaginary axis but also with respect to the real axis. Thus in particular the Jordan blocks associated with purely imaginary eigenvalues iα α > occur in complex conjugate pairs and it turns out that their signs in the sign characteristic are related. It can be shown that if J m1 (iα)...j mp (iα) are the Jordan blocks of a Hamiltonian matrix H associated with the eigenvalue iα and having the signs s 1...s p then the signs of the corresponding Jordan blocks J m1 ( iα)...j mp ( iα) are s 1... s p respectively. Another key difference between the real and the complex case is the behavior of the eigenvalue when H is singular. While in the complex case this eigenvalue can be treated as any other purely imaginary eigenvalue it has a special Jordan structure in the real case: each odd-sized Jordan block associated with zero must have an even number of copies and in the corresponding sign characteristic half of the sign must be +1 and half of the signs must be 1. In contrast there is no such pairing for Jordan blocks associated with zero that have even sizes. This extraordinary behavior of the eigenvalue zero leads to a real version of Theorem 3 that yields slightly different conditions in comparison with the complex case. Theorem 4 (28). A real Hamiltonian matrix H has a decomposition (5) with a real symplectic transformation matrix W if and only if for each nonzero purely imaginary eigenvalue H has an even number of odd-sized Jordan blocks half of which have sign +1 and half of which have sign 1. For most of the problems arising from systems and control one actually is interested in special invariant Lagrangian subspaces of Hamiltonian matrices. For instance for the existence of solutions of algebraic Riccati equations (24 38 55) one is interested in the invariant Lagrangian subspaces of a Hamiltonian matrix corresponding to the eigenvalues in the closed or open left half complex plane. A more general question is the following: if H is an 2n 2n Hamiltonian matrix and a list Λ of n of its eigenvalues (counted with multiplicities) is prescribed does there exists an invariant Lagrangian subspace associated with the eigenvalues in Λ and if so is this subspace unique? This question can be answered with the help of Theorem 3 or its corresponding real version. As we already know the existence of an invariant Lagrangian subspace for a Hamiltonian matrix H is equivalent to the existence of a decomposition of the form (5). From (5) the spectrum of H is the union of the spectra of both T and T. So one may assume that H has pairwise distinct non purely imaginary eigenvalues

12 Christian Mehl and Hongguo Xu λ 1 λ 1...λ p λ p with algebraic multiplicities ν 1 ν 1...ν p ν p and pairwise distinct purely imaginary eigenvalues iα 1...iα q with algebraic multiplicities 2µ 1...2µ q. In order to have an invariant Lagrangian subspace or equivalently a decomposition (5) it is necessary that the spectrum of T contains λ j and λ j with algebraic multiplicities k j and ν j k j respectively for each j = 1... p and µ j copies of iα j for each j = 1...q. Let Ω(H) denote the set of all possible spectra for T in a decomposition of the form (5) of H. Then this set contains p j=1 (ν j + 1) different selections because k j can be any number from to ν j for each j. Among them there are 2 p selections that contain either λ j or λ j but not both for all j. This subset of Ω(H) is denoted by Ω(H). The answer to the question of existence of invariant Lagrangian subspaces with a prescribed spectrum is then given in the following theorem. Theorem 5 (12). A Hamiltonian matrix H has an invariant Lagrangian subspace corresponding to every ω Ω(H) if and only if the conditions Theorem 3 (or Theorem 4 in real case) hold. Concerning uniqueness we have the following conditions. (i) For every ω Ω(H) H has a unique corresponding invariant Lagrangian subspace if and only if for every non purely imaginary eigenvalue λ j (and λ j ) H has only a single Jordan block and for every purely imaginary eigenvalue iα j H only has even-sized Jordan blocks all of them having the same sign. (ii) For every ω Ω(H) H has a unique corresponding invariant Lagrangian subspace if and only if for every purely imaginary eigenvalue iα j H has only evensized Jordan blocks all of them having the same sign. When the Lagrangian invariant subspaces corresponding to the eigenvalues in Ω(H) or Ω(H) are not unique then it is possible to parameterize their bases. Moreover the results in Theorem 5 can be used to study Hermitian solutions to algebraic Riccati equations see 12. We will now turn to skew-hamiltonian matrices. Analogously to the case of Hamiltonian matrices it can be shown that if the columns of W 1 span an invariant Lagrangian subspace of a skew-hamiltonian matrix K then there exists a symplectic matrix W = W 1 W 2 such that W 1 T D KW = T D = D. (6) Structured canonical forms for complex skew-hamiltonian matrices can be constructed in the same way as for complex Hamiltonian matrices using the fact that K is skew-hamiltonian if and only if ik is Hamiltonian. Thus the conditions for the existence of invariant Lagrangian subspaces are the same as in Theorem 3 replacing purely imaginary eigenvalues with real eigenvalues.

Canonical forms of structured matrices and pencils 13 Interestingly for any real skew-hamiltonian matrix the real version of the decomposition (6) always exists see 5. Also it is proved in 1 that for a real skew-hamiltonian matrix K there always exists a real symplectic matrix W such that W 1 KW = N N T where N is in real Jordan canonical form. The result shows clearly that every Jordan block of K has an even number of copies. Finally if S F n n is a symplectic matrix and if the columns of W 1 span an invariant Lagrangian invariant subspace of S then similar to the Hamiltonian case one can show that there exists a symplectic matrix W = W 1 W 2 such that W 1 T D SW = T DT = (DT ). (7) The case of symplectic matrices can be reduced to the case of Hamiltonian matrices with the help of the Cayley transformation see Chapter 2 in this book for details on the Cayley transformation. Therefore structured Jordan canonical forms for symplectic matrices can be derived using the structured canonical forms for Hamiltonian matrices in Theorem 2 and its real version. Then conditions for the existence of a decomposition of the form (7) can be obtained which are essentially the same as in Theorem 3 and 4 with purely imaginary eigenvalues replaced by unimodular eigenvalues in the symplectic case see 28. 4 Doubly structured matrices and pencils In this section we discuss canonical forms of doubly structured matrices and pencils. This research was mainly motivated by applications from quantum chemistry 2 3 16 41 49. In linear response theory one has to solve a generalized eigenvalue problem with a pencil of the form λ C Z Z C E F (8) F E where CEF are n n Hermitian matrices and Z is skew-hermitian. The simplest response function model is the time-dependent Hartree-Fock model (also called random phase approximation) in which the pencil (8) takes the simpler structure C = I and Z = so that the corresponding eigenvalue problem can be reduced to a standard eigenvalue problem with a matrix of the form E F A = F E

14 Christian Mehl and Hongguo Xu with E and F being Hermitian. It is straightforward to check that A is Hamiltonian (or J-skew-Hermitian) and M-Hermitian where In M =. I n In the general setting we consider matrices that are structured with respect to two invertible Hermitian or skew-hermitian matrices K and M. Because any skew- Hermitian matrix K can be transformed to the Hermitian matrix ik and any K- skew-hermitian matrix A can be transformed to the K-Hermitian matrix ia we may assume that both K and M are invertible and Hermitian and consider two cases only: (a) A is K-Hermitian and M-Hermitian i.e. KA = A K MA = A M (b) A is K-Hermitian and M-skew-Hermitian i.e. KA = A K MA = A M. The task is now to find an invertible matrix X to perform a transformation A = X 1 AX K = X KX M = X MX so that the canonical form of Corollary 1 (or the corresponding version for M-skew- Hermitian matrices) for both pairs (A K) and (A M) can simultaneously be recovered. As shown in 34 this is not always possible because the situation is too general. So it is reasonable to restrict oneself to the situation where the pencil λk M is nondefective meaning that all the eigenvalues of the Hermitian pencil λ K M are semisimple. (This assumption is satisfied in the case K = ij n and M = diag(i n I n ) which is relevant for the applications in quantum chemistry.) Then by (4) there exists an invertible matrix Q such that Q (λk M)Q = (λk 1 M 1 ) (λk p M p ) (9) where for each j = 1... p either λk j M j = λ 1 λ j 1 λ j containing a pair of nonreal eigenvalues λ j λ j or λk j M j = s j ( λ 1 α j ) s j = ±1 containing a single real eigenvalue α j with a sign s j. (We highlight that the same eigenvalues λ j λ j or α j respectively may appear multiple times among the blocks λk 1 M 1...λK p M p.) Under this assumption the following structured canonical form can be obtained for a matrix that is doubly structured in the sense of case (a) above.

Canonical forms of structured matrices and pencils 15 Theorem 6 (34). Suppose KM are Hermitian and invertible such that the pencil λk M is nondefective. Suppose A is both K-Hermitian and M-Hermitian. Then there exists an invertible matrix X such that A := X 1 AX = A 1 A 2 A p K := X KX = K 1 K 2 K p M := X MX = M 1 M 2 M p where for each j = 12... p the blocks A j K j M j are in one of the following forms. (i) Blocks associated with a pair of conjugate complex eigenvalues of A: Jm A j = j (λ j ) Rm K J m j ( λ j ) j = j γ j R m M R m j j = j γ j R m j where λ j C \ R γ j = c j + id j with c j d j R and d j. (ii) Blocks associated with real eigenvalues of A and real eigenvalues of λk M: A j = J n j (α j ) K j = s j R n j M j = s j η j R n j where s j = ±1 η j R and α j R. The sign of the block A j with respect to K is s j and the sign with respect to M is sign(s j η j ). (iii) Blocks associated with real eigenvalues of A and a pair of conjugate complex eigenvalues of λk M: Jn A j = j (α j ) Rn K J n j (α j ) j = j γ j R n M R n j j = j γ j R n j where α j R and γ j = c j + id j with c j d j R and d j >. Thus A j contains a pair of two n j n j Jordan blocks of A associated with the same real eigenvalue α j. The pairs of corresponding signs are (+1 1) with respect to both K and M. It is easily seen that the structured canonical forms for A with respect to K and M respectively can immediately be read off from the canonical form in Theorem 6. In addition the structured canonical form of λk M as in (9) can easily be derived from λk M. Therefore Theorem 6 combines three different structured canonical forms into one. On the other hand Theorem 6 shows that the presence of two structures in A leads to additional restrictions in the Jordan structure of A which can be seen from the blocks of type (iii) of Theorem 6. This block is indecomposable in a sense that there does not exist any transformation of the form (A j K j M j ) (Y 1 A j YY K j YY M j Y ) that simultaneously block-diagonalizes all three matrices. As a consequence the Jordan structure of a matrix A that is both K-Hermitian and M-Hermitian is rather restricted if the pencil λk M (is defective and) has only nonreal eigenvalues. In that case each Jordan block associated with a real eigenvalue of A must occur an even number of times in the Jordan canonical form of A. In particular all real eigenvalues of A must have even algebraic multiplicity.

16 Christian Mehl and Hongguo Xu In case (b) i.e. when A is K-Hermitian and M-skew-Hermitian then the eigenstructure of A has even richer symmetry than in case (a) because now the spectrum has to be symmetric to both the real and the imaginary axes. Also the Jordan blocks associated with real eigenvalues of A will have signs with respect to K while the ones associated with purely imaginary eigenvalues will have signs with respect to M. Thus the eigenvalue zero will play a special role because it will have signs both with respect to K and to M. A structured canonical form for this case will be given in the next theorem for which we need extra notation. By Σ n we denote the n n anti-diagonal matrix alternating sign matrix i.e. ( 1) Σ n =... ( 1) n 1 ( 1) 1 Theorem 7 (34). Suppose KM are Hermitian and invertible and λk M is nondefective. Suppose A is both K-Hermitian and M-skew-Hermitian. Then there exists an invertible matrix X such that. A := X 1 AX = A 1 A 2 A p K := X KX = K 1 K 2 K p M := X MX = M 1 M 2 M p where for each j = 12... p the blocks A j K j M j are in one of the following forms. (i) Blocks associated with nonreal non purely imaginary eigenvalues of A: A j = J m j (λ j ) J m j (λ j ) J m j ( λ j ) J m j ( λ j ) R m j K j = R m j M j = R m j R m j γ j R m j γ j R m j γ j R m j γ j R m j where λ j = a j + ib j with a j b j R and a j b j > and the parameter γ j satisfies one of the following three mutually exclusive conditions: (a) γ j = β j with β j > (b) γ j = iβ j with β j > or (c) γ j = c j + id j with c j d j R and c j d j >.

Canonical forms of structured matrices and pencils 17 (ii) Blocks associated with a pair of real eigenvalues ±α j of A and nonreal non purely imaginary eigenvalues of λk M: A j = J n j (α j ) J n j (α j ) J n j (α j ) J n j (α j ) R n j K j = R n j M j = R n j R n j γ j R n j γ j R n j γ j R n j γ j R n j where < α j R and γ j = c j +id j with c j d j R and c j d j >. The two Jordan blocks associated with α j have the signs 1 and 1 with respect to K and the two Jordan blocks associated with α j also have the signs 1 and 1 with respect to K. (iii) Blocks associated with a pair of real eigenvalues ±α j of A and real or purely imaginary eigenvalues of λk M: Jn A j = j (α j ) J n j (α j ) Rn j K j = s ) γ j R n j 2 M j = j Rn j γ j R n j ( γ j γ j where < α j R s j = ±1 γ j = β j or γ j = iβ j with < β j R. The Jordan block of A associated with α j has the sign s j with respect to K and the one associated with α j has the sign ( 1) n j+1 s j (γ j / γ j ) 2 with respect to K. (iv) Blocks associated with a pair of purely imaginary eigenvalues ±iα j of A and nonreal non purely imaginary eigenvalues of λk M: A j = K j = ij n j (α j ) ij n j (α j ) ij n j (α j ) ij n j (α j ) R n j R n j R n j R n j M j = γ j R n j γ j R n j γ j R n j γ j R n j where < α j R and γ j = c j +id j with c j d j R and c j d j >. The two Jordan blocks associated with iα j have the signs 1 and 1 with respect to M and the two Jordan blocks associated with iα j also have the signs 1 and 1 with respect to M.

18 Christian Mehl and Hongguo Xu (v) Blocks associated with a pair of purely imaginary eigenvalues ±iα j of A and real or purely imaginary eigenvalues of λk M: ijn A j = j (α j ) ij n j (α j ) Rn Rn j K j = j M R n j j = s j γ j ( ) γ j 2 γ Rn j j where < α j R s j = ±1 γ j = β j or γ j = iβ j with < β j R. The Jordan block of A associated with iα j has the sign s j with respect to M and the one associated with α j has the sign ( 1) n j+1 s j (γ j / γ j ) 2 with respect to M. (vi) A pair of blocks associated with the eigenvalue zero of A and nonreal non purely imaginary eigenvalues of λk M: Jn A j = j () Rn K J n j () j = j γ j Σ n M R n j j = s j j ( 1) n j+1 γ j Σ n j where s j = ±1 γ j = c j +id j with c j d j R and c j d j >. The two Jordan blocks of A associated with the eigenvalue have the signs 1 and 1 with respect to both K and M. (vii) A pair of blocks associated with the eigenvalue zero of A and real or purely imaginary eigenvalues of λk M: A j = Jn j () J n j () K j = Rn j R n j M j = γ j Σ n j γ j Σ n j where γ j = β j if n j is even and γ j = iβ j if n j is odd for some < β j R. The two Jordan blocks of A associated with the eigenvalue zero of A have the signs 1 and 1 with respect to both K and M. (viii) A single block associated with the eigenvalue zero of A and real or purely imaginary eigenvalues of λk M: A j = J n j () K j = s j R n j M j = σ j γ j Σ n j where s j σ j = ±1; and γ j = β j if n j is odd and γ j = iβ j if n j is even for some < β j R. The Jordan block of A associated with the eigenvalue zero has the sign s j with respect to K and the sign γ j γ j σ j i n j 1 with respect to M. Theorem 7 shows the intertwined connection of the three different structures: the double structure of A with respect to K and M and the structure of the Hermitian pencil λk M. The property of being K-Hermitian forces the spectrum of A to be symmetric with respect to the real axis and the property of being M-skew-Hermitian forces the spectrum to be symmetric with respect to the imaginary axis. The particular structure of blocks however depends in addition on the eigenvalues of the Hermitian pencil λk M. Interestingly there is not only a distinction between real and nonreal eigenvalues of λk M but also the purely imaginary eigenvalues of

Canonical forms of structured matrices and pencils 19 λk M play a special role. This effect can in particular be seen in the blocks associated with the eigenvalue zero the only point in the complex plane that is both real and purely imaginary. Depending on the type of the corresponding eigenvalues of λk M we have the following cases: a) real eigenvalues of λk M: in this case even-sized Jordan blocks of A associated with zero must occur in pairs (vii) but odd-sized Jordan blocks need not (viii); b) purely imaginary eigenvalues of λk M: in this case odd-sized Jordan blocks of A associated with zero must occur in pairs (vii) but even-sized Jordan blocks need not (viii); c) nonreal non purely imaginary eigenvalues of λk M: in this case all Jordan blocks of A associated with the eigenvalue zero must occur in pairs (vi). Structured canonical forms for A as a K-Hermitian matrix for A as an M- Hermitian matrix and for the Hermitian pencil λk M can be easily derived from the canonical form in Theorem 7 so again the result combines three different canonical forms into one. As the particular application from quantum chemistry shows there is also interest in doubly structured generalized eigenvalue problems. In general we can consider a matrix pencil λa B with both AB being doubly structured with respect to two invertible Hermitian or skew-hermitian matrices K and M. It turns out that a structured Weierstraß canonical form for a regular doubly structured pencil can easily be derived by using the results of the matrix case as in Theorems 6 and 7. Theorem 8 (34). Suppose K M are both invertible and each is either Hermitian or skew-hermitian i.e. K = σ K K M = σ M M σ K σ M = ±1. Let λa B be a regular pencil (that is det(λa B) ) with AB satisfying A K = ε A KA A M = δ A MA ε A δ A = ±1 B K = ε B KB B M = δ B MB ε B δ B = ±1. Then there exist invertible matrices XY such that Y 1 I H (λa B)X = λ E I X K1 KY = X M1 MY = K 2 M 2 where E is nilpotent and all three matrices in (HK 1 M 1 ) and (EK 2 M 2 ) respectively have the same sizes. Furthermore we have that K 1 = σ K ε A K 1 M 1 = σ M δ A M 1 ; H K 1 = (ε A ε B )K 1 H H M 1 = (δ A δ B )M 1 H K 2 = σ K ε B K 2 M 2 = σ M δ B M 2 ; E K 2 = (ε A ε B )K 2 E E M 2 = (δ A δ B )M 2 E.

2 Christian Mehl and Hongguo Xu Clearly H is a doubly structured matrix associated with the Hermitian or skew- Hermitian matrices K 1 M 1 and E is a doubly structured matrix associated with the Hermitian or skew Hermitian matrices K 2 M 2. Thus the pencil λa B is decoupled and becomes (λi H) (λe I). Hence a structured Weierstraß canonical form of λa B can be derived by applying the results in Theorems 6 and 7 to H and E separately. Note that in Theorem 8 one does not require λk M to be nondefective. However in order to apply Theorems 6 or 7 to obtain structured Jordan canonical forms for the matrices H and E the condition that both λk 1 M 1 and λk 2 M 2 are nondefective is necessary. Finally we point out that for the special type of doubly structured matrices and matrix pencils from linear response theory (16 41 49) necessary and sufficient conditions for the existence of structured Schur-like forms (obtained under unitary transformations) were provided in 36. 5 Structured singular value decompositions The singular value decomposition (SVD) is an important tool in Matrix Theory and Numerical Linear Algebra. For a given matrix A C m n it computes unitary matrices XY such that Y AX is diagonal with nonnegative diagonal entries. The condition that X and Y are unitary can be interpreted in such a way that the standard Euclidean inner product is preserved by the transformation with X and Y. Thus to be more precise we have a transformation on the matrix triple (AI n I m ) that yields the canonical form Y AX = X I n X = I n Y I m Y = I m where is diagonal with positive diagonal entries. But the singular value decomposition even yields more information as the nonzero singular values are the square roots of the positive eigenvalues of the matrices AA and A A. Thus in addition to a canonical form for A under unitary equivalence the SVD simultaneously provides two spectral decompositions Y (AA )Y = 2 X (A A)X = 2 for the Hermitian (positive semi-definite) matrices AA and A A. This concept can be generalized to the case of possibly indefinite inner products. Suppose that the two spaces C n and C m are equipped with inner products given by the Hermitian invertible matrices K C n n and M C m m respectively. Then the task is to find invertible matrices X and Y such that A = Y AX K = X KX M = Y MY (1)

Canonical forms of structured matrices and pencils 21 are in a canonical form so that also the canonical forms of the K-Hermitian matrix T and the M-Hermitian matrix Z can easily be derived where T = K 1 A M 1 A Z = M 1 AK 1 A. (11) Equivalently we obtain structured canonical forms for the two Hermitian pencils λk A M 1 A and λm AK 1 A The transformation (1) has several mathematical applications. For instance the existence of a generalization of a polar decompositions for a matrix A in a space equipped with an indefinite inner product as in (1) given by the invertible Hermitian matrix M C n n is related to the matrices AA and A A where A := M 1 AM. By definition a matrix A C n n is said to have an M-polar decomposition if there exists an M-Hermitian matrix H and an M-unitary matrix U such that A = UH see 5 6. In contrast to the classical polar decomposition in the case of the Euclidean inner product an M-polar decomposition need not exist for a given matrix A C n n. In 37 it was proved that a matrix A C n n allows an M-polar decomposition if and only if the two M-Hermitian matrices AA and A A have the same canonical forms (as in Corollary 1) - a fact that was already conjectured in 18. If a canonical form under a transformation as in (1) is given with the matrices T and Z as in (11) then we have that AA = MT M 1 and A A = Z with K = M 1. Thus structured canonical forms can easily be derived from the canonical form under the transformation (1). On the other hand the simultaneous transformation (1) provides more flexibility in solving the eigenvalue problem of a structured matrix as B = A M 1 A from a numerical point of view. That is instead of performing similarity transformations on B one may use two-sided transformations on A. For example when K = J and M = I a structured condensed form for a matrix A was proposed in 53 and a numerical method was given in 54. In the case when A is invertible (hence square) the following theorem provides the desired canonical form for the transformation in (1). Here we use the notation J 2 m(α) for the square of a Jordan block J m (α) of size m associated with the eigenvalue α. Theorem 9 (35). Let A C n n be nonsingular and let KM C n n be Hermitian and invertible. Then there exist invertible matrices XY C n n such that Y AX = A c A r X KX = K c K r Y MY = M c M r. (12) Consequently for the K-Hermitian matrix T = K 1 A M 1 A and the M-Hermitian matrix Z = M 1 AK 1 A one has X 1 T X = T c T r Y 1 ZY = Z c Z r. (13) The diagonal blocks in (12) and (13) have the following forms.

22 Christian Mehl and Hongguo Xu (i) Jm1 (µ A c = 1 ) Jmp (µ p ) J m1 ( µ 1 ) J mp ( µ p ) Rm1 Rmp K c = R m1 R mp Rm1 Rmp M c = R m1 R mp J 2 T c = m1 (µ 1 ) J 2 Jm 2 mp (µ p ) 1 ( µ 1 ) Jm 2 p ( µ p ) J 2 Z c = m1 (µ 1 ) J 2 Jm 2 mp (µ p ) 1 ( µ 1 ) Jm 2 p ( µ p ) where µ j = a j + ib j with < a j b j R for j = 1... p. For each j both the diagonal block diag ( J 2 m j (µ j )J 2 m j ( µ j ) ) of T c as well as the diagonal block diag ( Jm 2 j (µ j )Jm 2 j ( µ j ) ) of Zc are similar to a matrix consisting of two m j m j Jordan blocks one of them associated with the nonreal and non purely imaginary eigenvalue µ 2 j and the other one with µ 2 j. (ii) A r = J n1 (β 1 ) J nq (β q ) K r = s 1 R n1 s q R nq M r = σ 1 R n1 σ q R nq T r = s 1 σ 1 Jn 2 1 (β 1 s q σ q Jn 2 q (β q ) ( Z r = s 1 σ 1 J 2 n1 (β 1 ) ) ( sq σ q J 2 nq (β q ) ) where β j > and s j σ j = ±1 for j = 1...q. For each j the block s j σ j J 2 n j (β j ) of T r is similar to an n j n j Jordan block associated with a real eigenvalue s j σ j β 2 j of T with the sign with respect to K being { s j if n j is odd or if n j is even and s j σ j = 1 σ j if n j is even and s j σ j = 1 and the block s j σ j ( J 2 n j (β j ) ) of Zr is similar to an n j n j Jordan block associated with a real eigenvalue s j σ j β 2 j of Z with the sign with respect to M being { σ j if n j is odd or if n j is even and s j σ j = 1 s j if n j is even and s j σ j = 1. For a general rectangular matrix A C m n the situation is more complicated because of (a) the rectangular form of A and (b) the presence of the eigenvalue in T or Z. Indeed note that these two matrices T and Z can be represented as products of the same two factors but with different order i.e. T = BC and Z = CB where B = K 1 A and C = M 1 A. By a well-known result 11 the Jordan structures of the nonzero eigenvalues of the two matrix products BC and CB are identical while

Canonical forms of structured matrices and pencils 23 this is not the case for the eigenvalue zero. Despite this additional complexity in the problem of finding a canonical form under the transformation (1) a complete answer is still possible as shown in the next theorem. Theorem 1 (35). Let A C m n and let K C n n and M C m m be Hermitian and invertible. Then there exist invertible matrices Y C m m and X C n n such that Y AX = A c A r A 1 A 2 A 3 A 4 X KX = K c K r K 1 K 2 K 3 K 4 (14) Y MY = M c M r M 1 M 2 M 3 M 4. Moreover for the K-Hermitian matrix T = K 1 A M 1 A C n n and for the M- Hermitian matrix Z = M 1 AK 1 A C m m we have that X 1 T X = T c T r T 1 T 2 T 3 T 4 Y 1 ZY = Z c Z r Z 1 Z 2 Z 3 Z 4. The blocks A c A r K c K r M c M r have the same forms as in (12). Therefore the blocks T c T r and Z c Z r have the same forms as in (13). The remaining blocks are associated with the eigenvalue of T and Z and have the following forms. (i) A 1 = l k K 1 = diag(i k1 I k2 ) M 1 = diag(i l1 I l2 ) T 1 = k Z 1 = l where k 1 + k 2 = k and l 1 + l 2 = l. So there are k copies of 1 1 Jordan blocks associated with the eigenvalue of T such that k 1 of them have the sign +1 and k 2 of them the sign 1 with respect to K and there are l copies of 1 1 Jordan blocks associated with the eigenvalue of Z such that l 1 of them have the sign +1 and l 2 of them the sign 1. (ii) A 2 = J 2r1 () J 2r2 () J 2ru () K 2 = R 2r1 R 2r2 R 2ru M 2 = R 2r1 R 2r2 R 2ru T 2 = J 2 2r 1 () J 2 2r 2 () J 2 2r u () Z 2 = ( J 2 2r 1 () ) T ( J 2 2r2 () ) T ( J 2 2ru () ) T. For each j = 1...u the block J 2 2r j () of T 2 is similar to a matrix consisting of two copies of the Jordan block J r j () of T with one of them having the sign +1 and the other having the sign 1 with respect to K and the block ( J 2 2r j () ) T is similar to a matrix consisting of two copies of the Jordan block J r j () of Z with one of them having the sign +1 and the other having the sign 1 with respect to M.