Proceedings of the Sixteenth International Symposium on Mathematical Networks and Systems (MTNS 2004), Leuven, Belgium Generalized Shifted Inverse Iterations on Grassmann Manifolds 1 J. Jordan α, P.-A. Absil β and R. Sepulchre γ α Department of Mathematics, University of Würzburg, 97074 Würzburg, Germany. jordan@mathematik.uni-wuerzburg.de β School of Computational Science and Information Technology, Florida State University, Tallahassee FL 32306-4120, USA. absil@csit.fsu.edu γ Department of Electrical Engineering and Computer Science, Institut Monteore, B28 Université de Liège, B-4000 Liège, Belgium. r.sepulchre@ulg.ac.be Abstract: We discuss a family of feedback maps for the generalized Inverse Iterations on the Grassmann manifold. The xed points of the resulting algorithms correspond to the eigenspaces of a given matrix. A sucient condition for local convergence is given. 1 Introduction In many applications it is necessary to nd a p-dimensional eigenspace of a given matrix A. There exist several dierent strategies to design algorithms for eigenspace computation, see for example the approaches in [7, 8, 10]. A classical and very successful algorithm for the case p = 1 and A = A T is the Rayleigh quotient iteration (RQI). Its dynamics can be described on the projective space, see for example [4, 9, 11]. A block version of the RQI method for 1 p < n was proposed in [3]. The iteration was shown to induce an iteration on the Grassmann manifold (i.e. the set of p-dimensional subspaces of R n ) and was therefore called Grassmann- RQI. Assuming A = A T, the Grassmann-RQI is locally cubic convergent to a p-dimensional invariant subspace of A. 1 This paper presents research partially supported by the Belgian Program on Interuniversity Poles of Attraction, initiated by the Belgian State, Prime Minister's Oce for Science, Technology and Culture. This work was completed while the rst author was a guest at the University of Liege under a grant from the Control Training Site (CTS). The second author's work was supported by the National Science Foundation of the USA under Grant ACI0324944 and by the School of Computational Science and Information Technology of Florida State University.
2 The Grassmann-RQI can be interpreted as a shifted Inverse Iteration on the Grassmann manifold with a certain feedback control. In this paper we want to generalize this idea by using dierent feedback strategies instead of the Rayleigh quotient. We introduce a set of feedback laws which ensure that the corresponding algorithm is well-posed on the Grassmann manifold. Similar to the Grassmann-RQI eigenspaces are corresponding to the xed points of the algorithm. Furthermore we prove local convergence for a certain set of algorithms. The paper is organized as follows. In Section 2 we generalize the Grassmann- RQI. Therefore we introduce a set of feedback maps. In Section 3 we discussed the algebraic structure of the feedback maps. Section 4 deals with the singularities of the algorithm. The correspondence between xed points and eigenspaces is discussed in Section 5. We give sucient criteria for local convergence in Section 6. Finally, we give some concluding remarks in Section 7. 2 The generalized shifted Inverse Iteration The Grassmann Rayleigh Quotient Iteration described in [1, 3] is a subspace iteration on the Grassmann manifold Grass(p, n). It can can be interpreted as a discrete-time system with a certain feedback control. In this section we introduce a family of feedback controls which generalizes this idea. Note that we do not assume symmetry of the matrix A in this section. We use the following notation. With ST(p, n) we denote the set of real full rank n by p matrices. Note that ST(p, n) is open in R n p and has therefore, in a canonical way, a smooth manifold structure (called the noncompact Stiefel manifold). Let π be the canonical projection π : ST(p, n) Grass(p, n) that maps Y ST(p, n) to its column space. For any X, Y ST(p, n) exists M GL p (R) such that X = Y M if and only if π(x) = π(y ). Given an initial iterate Y 0 Grass(p, n), the Grassmann-RQI [3] computes a sequence of subspaces Y t = Φ R (Y t 1 ), t = 1,..., t final, where Φ R is dened as follows. Algorithm 2.1 (Grassmann-RQI mapping Φ R ) Given Y Grass(p, n), 1) Choose Y π 1 (Y), i.e. a matrix Y ST(p, n) with π(y ) = Y. 2) Solve the Sylvester equation AY + Y + R(Y ) = Y (1) with the Rayleigh quotient map R : Y (Y T Y ) 1 Y T AY. 3) Dene Φ R (Y) := Y + := π(y + ). The map Φ R is well dened under the following assumptions: (i) the Sylvester equation (1) admits one and only one solution, (ii) this solution has full rank and (iii) Y + is independent of the choice of Y π 1 (Y). One can show that assumptions (i) and (ii) holds for an open and dense subset of matrices Y ST(p, n).
3 Moreover, if (i) and (ii) are fullled then (iii) is fulllled as well. This is due to the following homogeneity property of R X ST(p, n), M GL p (R) : R(XM) = M 1 R(X)M. (2) We call a map F : ST(p, n) GL p (R) with Property (2) a feedback map and denote the set of all feedback maps with F. For any F F we dene the iteration mapping Φ F as follows. Algorithm 2.2 (Grassmann shifted Inverse Iteration mapping Φ F ) Given Y Grass(p, n), 1) Choose Y π 1 (Y). 2) Solve the Sylvester equation 3) Dene Φ F (Y) := Y + := π(y + ). AY + Y + F (Y ) = Y (3) We denote the corresponding iterative algorithm Y k = Φ F (Y k 1 ), the Generalized Shifted Inverse Iteration. Note that Property (2) ensures that the new iterate Y + does not depend on the choice of Y in algorithm step 1. It is possible to choose a time varying F F. This leads to a discrete-time control system Y 0 Grass(p, n), Y t+1 = Φ(F t, Y t ), F t F. (4) In this paper we consider algorithms of type (2.2) with xed F F. But the dynamic properties of (4) will be the aim of future work. 3 The algebra of feedback maps We have dened the Generalized Shifted Inverse Iteration for any feedback map F F. Obviously F is not empty, since the Rayleigh quotient map belongs to F. In this section we give some more examples and show that F has a rich algebraic structure. Theorem 3.1 With multiplication F G : X F (X)G(X), addition F + G : X F (X) + G(X), scalar multiplication λf : X λf (X), zero element X 0 R p p and one element X I R p p, F is a real algebra. Using Theorem (3.1) is not dicult to construct examples of feedback maps. In particular every algebraic combination of the following examples is an element of F. Examples 3.2 1) F 0. This choice of F leads to the (unshifted) Inverse Iteration on Grass(p, n).
4 2) The Rayleigh quotient map R : X (X T X) 1 X T AX is an element of F. Note that F := R gives Algorithm 2.1. 3) For W R n p the map F W : X (W T X) 1 W T AX dened for all X with W T X GL p (R) is in F. 4) Let F be a feedback map and f : ST(p, n) R be a map with f(xm) = f(x) for all X ST(p, n) and all M GL p (R). Then the map ff : X f(x)f (X) is an element of F. 5) Let B : ST(p, n) R n n be a map with B(XM) = B(X) for all X ST(p, n) and all M GL p (R). Then the map F B : X (X T X) 1 X T B(X)X is also element of F. One can construct an innite set of linear independent maps F α F. Therefore F has innite dimension as a vector space. 4 Singularities of Φ F In general, there may exist subspaces Y Grass(p, n) for which Φ F (Y) is not a well-dened element of Grass(p, n). This happens if and only if either Equation (3) fails to have an unique solution, or the unique solution fails to have full rank. Remarkably, under convenient conditions on F, Φ F is well dened on a generic subset of Grass(p, n). With M F we denote the set of all matrices Y ST(p, n) such that Equation (3) has a unique solution and this solution has full rank. If X ST(p, n) and Y ST(p, n) represent the same element of Grass(p, n) (i.e. π(x) = π(y )) then X M F if and only if Y M F. Therefore, the question if Φ F is well dened or not, does not depend on the choice of the representation Y of Y Grass(p, n). A reasonable assumption for F is to be a rational functions of the entries y ij of Y ST(p, n) as in the case of the generalized Rayleigh quotient map. Our results also hold for a wider class of feedback maps. We call a continuous map F : A B quasi open if for every S A with nonempty interior F (S) has nonempty interior in B. Theorem 4.1 Let F : R n p R p p rational or quasi open and continuous on ST(p, n). a) The set of matrices Y for which Equation (3) has a unique solution is open and dense, unless F λ p I for any eigenvalue λ p of A. b) π(m F ) is either open and dense in Grass(p, n), or empty. Note that the case π(m F ) = is rather exceptional and easy to verify. particular this is the case if F λ p I for any eigenvalue λ p of A. In 5 Correspondence between xed points and eigenspaces If the feedback law F is an element of F, then the xed points of Algorithm 2.2 are related to the eigenspaces of A. Let V be a xed point of the map
5 Φ F : Y Y +, then there exits M GL p (R) such that AY M Y MF (Y ) = Y. (5) Using Property (2) we get AY = Y P with P = (M 1 + F (Y M 1 )) R p p. Thus, Aπ(Y ) π(y ). Theorem 5.1 If Y Grass(p, n) is a xed point of Φ F then Y is an eigenspace of A. Conversely, if Y is an eigenspace of A, then Y is a xed point of Φ F provided that Y π(m F ). Observe that the unshifted algorithm (i.e. the choice F O) reduces to the Inverse Iteration. In this case the set of xed points and the set of eigenspaces of A coincide. In the shifted algorithm no new xed point is created but some eigenspaces may become singularities. This is for instance the case of the Grassmann-RQI. Its very nature makes every eigenspace a singularity of the algorithm, thereby accelerating the rate of convergence. Nevertheless, the Grassmann-RQI mapping Φ R has a continuous extension such that xed points of the extended map coincide with the eigenspaces. 6 Local convergence In the following we want to state a sucient condition on F which guarantees local convergence of Algorithm 2.2 for symmetric matrices A. To measure distances on Grass(p, n) we use d(x, Y) := Π X Π Y 2 where Π X denotes the orthogonal projection on X. Note that the topology induced on Grass(p, n) by the distance d(x, Y) is identical to the one induced by the canonical projection π : ST(p, n) Grass(p, n) (see [6]). Because the following theorem is stated in local coordinates we need some terminology and properties of the geometry on Grass(p, n). Let X Grass(p, n) be a xed element. We choose an orthogonal X π 1 (X ) and X R (n p) p such that Q := (X X ) O n (R). Furthermore, we use the notation ( ) Q T A11 A AQ = 12. (6) A 21 A 22 X is called spectral (with respect to A) if A 11 R p p and A 22 R (n p) (n p) have no eigenvalues in common. For Y Grass(p, n) which is not orthogonal to X (i.e. X T Y GL p (R) for Y π 1 (Y)), pick Ỹ π 1 (Y) and dene σ X (Y) = Ỹ (X T Ỹ ) 1. One easily veries that σ X (Y) is independent of the choice Ỹ π 1 (Y). Thus, the map K X : Grass(p, n) R (n p) p, Y X T σ X(Y) is well dened. Note that K X denes a coordinate chart for Grass(p, n). The distance of a point Y Grass(p, n) which is not orthogonal to X can be approximated in terms of the local coordinate K X (Y) by d(x, Y) = K X (Y) 2 + O( K X (Y) 3 2). (7)
6 For a deeper introduction to the geometry on Grass(p, n) see [1, 3, 2, 10]. The following theorem gives a sucient condition for local convergence of Algorithm 2.2. Theorem 6.1 Let A be a symmetric n by n matrix and X be a p-dimensional spectral eigenspace of A. Let X π 1 (X ) be orthogonal and θ > 0 a constant. Let F F be continuous with property F (σ X Y) X T AX 2 = O( K X (Y) θ 2), (8) for all Y in a neighborhood of X. The Grassmann shifted Inverse Iteration mapping Φ F admits a continuous extension on a neighborhood of X. The point X is an attractive xed point of the extended mapping, and the rate of convergence is θ + 1. In particular Theorem 6.1 gives locally cubic convergence for the Rayleigh quotient R(Y ) = (Y T Y ) 1 Y T AY. This result was already proved in [1]. It is possible to construct other maps F which fulll the conditions of the theorem. Nevertheless Condition (8) is certainly a very hard restriction on the choice of F. If one wants to apply Theorem 6.1 to prove cubic convergence close to a certain eigenspace X, F has to behave locally like the Rayleigh quotient map. On the other hand, since we have a freedom in the choice of F F, Algorithm 2.2 may open new possibilities to improve the global behavior of the iteration. 7 Conclusion and future work Given a matrix A R n n we have constructed a family of iterations dened on suciently large subsets of Grass(p, n). The xed points of the algorithms correspond to the p-dimensional eigenspaces of A. Therefore these algorithms may be used for eigenspace calculations. Furthermore, we state a condition for local convergence to the xed points. The Grassmann-RQI can be seen as a particular case with a constant control. In the case p = 1 (i.e. Grass(p, n) = RP n 1 ) the Grassmann-RQI reduces to the well-known Rayleigh quotient iteration which is an ideal shift-strategy in a certain sense and has some useful global properties ([4, 11]). In our future work we want to study control systems of type (4). In particular we want toinvestigate if the generalized Rayleigh quotient map R is (locally) an ideal choice compared with other possible shifts. Furthermore we want to construct feedback strategies which improve the global behavior. References [1] P.-A. Absil, Invariant Subspace Computation: A Geometric Approach, PhD Thesis, Liege (2003). [2] P.-A. Absil and R. Mahony and R. Sepulchre, Riemannian geometry of Grassmann manifolds with a view on algorithmic computation, Acta Appl. Math. 80, No 2, (2004), pp 199220.
REFERENCES 7 [3] P.-A. Absil, R. Mahony, R. Sepulchre and P. Van Dooren, A Grassmann-Rayleigh Quotient Iteration for Computing Invariant Subspaces, SIAM Review, 44, No 1, (2002), pp. 57-73. [4] S. Batterson and J. Smillie, The dynamics of Rayleigh quotient iteration, SIAM J. Numer. Anal., 26 (1989), pp. 624-636. [5] S. Batterson and J. Smillie, Rayleigh quotient iteration for nonsymmetric matrices, Math. Comp., 55, No 191,(1990), pp 169-178. [6] J. Ferrer, M.I. García, and F. Puerta, Dierential families of subspaces, Linear Algebra Appl., 199(1994), pp. 229-252. [7] U. Helmke, J. Moore, Optimization and Dynamical Systems, Springer- Verlag, New York, (1994). [8] K. Hüper, A calculus approach to matrix eigenvalue algorithms, Habilitationsschrift, Würzburg (2002). [9] I. C. F. Ipsen, Computing an eigenvector with inverse iteration, SIAM Rev. 39 (1997), pp. 254-291. [10] G.W. Stewart, Error and perturbation bounds for subspace associated with certain eigenvalue problems, Siam Review, 15, No 4, (1973), pp. 727-764. [11] B. N. Parlett, The Rayleigh Quotient Iteration and Some Generalizations for Nonnormal Matrices, Mathematics of Computation, 28, No 127, (1974), pp 679-693. [12] P. Van Dooren, R. Sepulchre, 'Shift policies in QR like algorithms and feedback control of self-similar ows', Open Problems in Mathematical Systems and Control Theory (V. Blondel, E. Sontag, M. Vidyasagar and J.C. Willems; Eds.), Springer, London (1999), pp. 245-250.