UMIAS-TR-9-5 July 99 S-TR 272 Revised March 993 Perturbation Theory for Rectangular Matrix Pencils G. W. Stewart abstract The theory of eigenvalues and eigenvectors of rectangular matrix pencils is complicated by the fact that arbitrarily small perturbations of the pencil can cause them disappear. However, there are applications in which the properties of the pencil ensure the existence of eigenvalues and eigenvectors. In this paper it is shown how to develop a perturbation theory for such pencils. Department of omputer Science and Institute for Advanced omputer Studies, University of Maryland, ollege Park, MD 2742. This work was supported in part by the Air Force Oce of Scientic Research under ontract AFOSR-87-88. This report is available by anonymous ftp from thales.cs.umd.edu in the directory pub/reports.
Perturbation Theory for Rectangular Matrix Pencils G. W. Stewart ASTRAT The theory of eigenvalues and eigenvectors of rectangular matrix pencils is complicated by the fact that arbitrarily small perturbations of the pencil can cause them disappear. However, there are applications in which the properties of the pencil ensure the existence of eigenvalues and eigenvectors. In this paper it is shown how to develop a perturbation theory for such pencils. In this note we will be concerned with the perturbation theory for the generalized eigenvalue problem Ax = x; () where A and are m n matrices with m > n and and are normalized so that jj 2 + jj 2 = : Although rectangular matrix pencils have a long history and a well developed theory of canonical forms (e.g., see [, 5, 6]), their eigenvalues and eigenvectors have been less well studied. There are two reasons. In the rst place, eigenvalues and eigenvectors may fail to exist. For example, if A = and = ; then () has no nontrivial solution unless =. In the second place, even if () has a solution, an arbitrarily small perturbation can make it go away, as the above example also shows. It is not easy to construct a general perturbation theory for objects that may not exist and can vanish at the drop of a hat. Nonetheless, in some applications the nature of the matrices A and insure the existence of eigenvectors, and the conditions that do so are stable under small perturbations (for an example from game theory, see [3]). In these circumstances The homogeneous form given here seems preferable to the more conventional form Ax = x. We call the pair (; ) an eigenvalue of the problem.
2 Rectangular Matrix Pencils it is reasonable to try to develop a perturbation theory. In this note we shall show how to do so by reducing the problem to a square one. We begin by describing the space of solutions of the unperturbed problem. First note that equation () says that Ax and x lie in the same one-dimensional subspace. Generalizing this observation, we say that a subspace X of dimension k is an eigenspace of the pencil A? if there is a subspace Y of dimension k such that AX + X Y: (2) Here the sums and products are the usual Minkowski operations; e.g., AX = f Ax : x 2 X g. Since the sum of two eigenspaces is an eigenspace, there is a unique maximal eigenspace, which we shall call the -eigenspace of the pencil. In what follows X will be the -eigenspace of the pencil A? and Y will be the corresponding subspace in (2). We will assume that X has dimension k >. The relation between eigenspaces and eigenvectors can be seen as follows. Let X = (X X 2 ) and Y = (Y Y 2 ) be unitary matrices such that R(X ) = X and R(Y ) = Y. Then Y H AX and Y H X have the forms Y H AX = A A 2 A 22 and Y H X = 2 : (3) 22 It follows that if x is an eigenvector of the pencil A?, then X x is an eigenvector of (). The converse is also true: all eigenvectors of () have the form X x. This is a consequence of the fact that the pencil A 22? 22 does not have an eigenspace. For if it did, we could reduce A 22 and 22 as above, so that the transformed pencil assumes the form A ^A2 ^A3 ^A22 ^A23 ^A33 A? ^2 ^3 ^22 ^23 ^33 A : Then the original pencil has an eigenspace of dimension greater than k corresponding to the pencil A ^A2 ^2? ; ^A22 ^22
Rectangular Matrix Pencils 3 which contradicts the maximality of the -subspace. In what follows, we will assume that the matrices of the pencil A? are in the reduced form (3). Note that since the reduction is a unitary equivalence any perturbation in the original pencil corresponds (in a unitarily invariant norm) to a perturbation of the same size in the reduced form. Let us now consider the perturbed matrices and where A = = A A2 E 2 A22 A + E A 2 + E 2 E 2 A 22 + E 22 2 F 2 22 + F 2 + F 2 ; F 2 22 + F 22 ke ij k; kf ij k ; i; j = ; 2: (4) Here k k denotes both the Euclidean vector norm and the subordinate spectral matrix norm. Let x = (x H x H 2 ) H be a normalized eigenvector of the corresponding pencil: Ax = x; kxk = : We are going to show that under a natural condition (namely, that dened below is not small) the second component x 2 of x is small. First note that since the pencil A 22? 22 has no eigenvectors, there is a number > such that k A 22 x 2? 22 x 2 k kx 2 k: (5) Since jj 2 + j j 2 =, if then Since x is an eigenvector? p 2 > ; (6) k A 22 x 2? 22 x 2 k kx 2 k: = k (E 2 x? A 22 x 2 )? (F 2 x? 22 x 2 )k kx 2 k? k E 2 x? F 2 x k: Hence kx 2 k p 2 :
4 Rectangular Matrix Pencils Now let us perform the reduction procedure describe above on the perturbed pencil, but using only the one-dimensional space spanned by the eigenvector x. Specically, let X = R Q P S be a unitary matrix whose rst column is x and for which kp k; kqk kx 2 k p 2 : (7) (The existence of of such a matrix is established in the appendix.) Let Y = be a unitary matrix whose rst column is proportional to Ax (or x if Ax = ). Then the rst k columns of Y H AX and Y H X have the forms a H A E A ^R ^P and ^Q ^S b H F A : (8) Thus (; ) is an eigenvalue of the pencil a H A? b H ut by direct computation, this pencil is or ( ^RH A R + ^RH E R + ^RH A2 P + ^QH E 2 R + ^QH A22 P )? where from (4) and (7) ( ^R H R + ^R H F R + ^R H A2 P + ^Q H F 2 R + ^Q H A22 P ) ( ^R H A R + G)? ( ^R H R + H); kgk; khk 2 + 2 p 2 maxfk Ak; k k :
Rectangular Matrix Pencils 5 We have thus shown that the eigenvalue (; ) is an eigenvalue of a perturbation of the pencil ^R H A R? ^R H R, a pencil which corresponds to the -eigenspace of the original problem. Since the pencil and its perturbation are square, we may use standard perturbation theory to bound the perturbation in the eigenvalue and its eigenvector (for a survey of the perturbation theory of the generalized eigenvalue problem, see [4, h. VI]). The dependence of the bounds on dened by (5) is entirely natural. If is small, then (; ) is almost an eigenvalue of the pencil A 22? 22 and could have come from the perturbed pencil A 22? 22. Such an eigenvalue need not be near an eigenvalue associated with the -eigenspace of the original problem. Seen in this light, the condition (6) insures that the perturbed eigenvalue truly comes from the -eigenspace of the original problem. Finally, we note that the general technique we have outlined here is at least as important as the particular bounds. For example, it can be extended to obtain perturbations bounds on eigenspaces. Again, if we work with nonorthogonal diagonalizing transformations, so that A 2 = 2 = in (3) and a H = b H = in (8), we obtain the result that and / + y H Ex + O(maxfkEk; kf kg 2 ) / + y H F x + O(maxfkEk; kf kg 2 ); where y H is the left eigenvector corresponding to (; ). The details are left to the reader. Appendix The following extension theorem is more general than we need but may be of independent interest. Let p q and let the matrix X = q p X n?p X 2 have orthonormal columns. Then there is a unitary matrix q p?q n?p p X X = X 2 X 3 n?p X 2 X 22 X 23
6 Rectangular Matrix Pencils such that k(x 2 X 22 )k = kx 2 k: To establish the theorem, rst assume that p+q n. y the S decomposition (e.g., see [2, p.77]) we may assume that X = q q p?q q n?p?q S A ; where and S are nonnegative diagonal matrices. Then the required extension is in which we take X = q p?q q n?p?q q?s p?q I q S n?p?q I (X 2 X 22 ) = S On the other hand, if p + q > n, we take X in the form X = In this case the required extension is p+q?n n?p p+q?n I n?p p?q n?p S : A A ; in which X = p+q?n n?p p?q n?p p+q?n I n?p?s p?q I A ; n?p S (X 2 X 22 ) = ( S ):
Rectangular Matrix Pencils 7 References [] F. R. Gantmacher (959). The Theory of Matrices, Vols. I, II. helsea Publishing ompany, New York. [2] G. H. Golub and. F. Van Loan (989). Matrix omputations. Johns Hopkins University Press, altimore, Maryland, 2nd edition. [3] E. Marchi, J. A. Oviedo, and J. E. ohen (99). \Pertrubation Theory of a Nonlinear Game of von Neuman." SIAM Journal on Matrix Analysis and Applications, 2, 592{596. [4] G. W. Stewart and G.-J. Sun (99). Matrix Perturbation Theory. Academic Press, oston. [5] P. Van Dooren (979). \The omputation of Kronecker's anonical Form of a Singular Pencil." Linear Algebra and Its Applications, 27, 3{4. [6] J. H. Wilkinson (979). \Kronecker's anonical Form and the QZ Algorithm." Linear Algebra and Its Applications, 28, 285{33.