An efficient algorithm to compute the real perturbation values of a matrix

Size: px

Start display at page:

Download "An efficient algorithm to compute the real perturbation values of a matrix"

Doreen Cameron
6 years ago
Views:

1 An efficient algorithm to compute the real perturbation values of a matrix Simon Lam and Edward J. Davison 1 Abstract In this paper, an efficient algorithm is presented for solving the nonlinear 1-D optimization problem associated with computing the real perturbation values of an arbitrary complex matrix M C q l. The real perturbation values of a matrix is important in computing the real stability radius, the real controllability radius, the real decentralized fixed-mode radius, etc. of the control literature. This is because obtaining these radii requires solving a one or even two dimensional optimization problem involving real perturbation values. Hence, being able to quickly compute the real perturbation values of a matrix is crucial in such calculations. A numerical example is included to demonstrate the effectiveness of the proposed algorithm. 2 Introduction The real perturbation values of a general complex matrix were first introduced in [1]. From a perturbation theory point of view, real perturbation values are analogous to singular values except it focuses only on real perturbations, which arises in problems such as parametric robustness analysis in control theory, and the computation of real pseudospectra in numerical analysis. As examples of where such perturbation problems can arise, one is often This work has been supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada under grant No. A4396. S. Lam and E. J. Davison is with the Systems Control Group, Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada M5S 3G4 {simon,ted}@control.utoronto.ca

2 interested in the control literature in studying the robustness of various system properties with respect to parametric perturbations. Consider the following linear time-invariant (LTI) multivariable system ẋ = Ax + Bu y = Cx + Du (1) where x R n, u R m, and y R r are respectively the state, input, and output vectors, and A, B, C, and D are constant matrices with the appropriate dimensions for n 1, m 1, r 1, and max(r, m) n. It is well known that there exists a LTI controller that can assign the eigenvalues of the closed-loop system to any arbitrary spectrum if and only if the system is controllable and observable. However, when a controllable and observable system is subject to parametric perturbations (i.e. A A + A, B B + B, C C + C, and D D + D ), the system may be very close to becoming uncontrollable and/or unobservable. Hence, a continuous controllability (observability) measure, called the controllability (observability) radius [3, 4], is more informative than the traditional yes/no controllability (observability) metric, which simply determines whether a system is controllable (observable) or not. The same can be said about other system properties such as having decentralized fixed modes (DFM) [6], stability [8], minimum-phase [7], etc. Computing the real controllability radius (i.e. when the perturbations are real, A R n n, B R n m, C R r n, and D R r m ) requires optimizing in the complex plane the real perturbation values of a complex matrix. Hence, being able to compute the real perturbation values, which itself is a 1-D optimization problem, will improve the computation of the real controllability radius. The same is also true in computing the real DFM radius [6], the real stability radius [8], the real minimum-phase radius [7], etc. An algorithm is proposed in this paper to efficiently compute the real perturbation values of a matrix. The paper is organized as follows. In Section 3, the definition and formula for computing the real perturbation values of a matrix are reviewed. Then in Section 4, a few useful tools are developed, followed by a thorough description of the proposed algorithm in Section 5. Finally, Section 6 gives a numerical example to demonstrate the effectiveness of the algorithm. 3 Notation and Background In this paper, the field of real and complex numbers are denoted by R and C respectively. The i-th singular value of a matrix M C p m is denoted by σ i (M), where σ 1 (M) σ 2 (M). M denotes the spectral norm of a matrix M and is equal to σ 1 (M). Also, M, M T, and M H denote respectively the complex conjugate, transpose, and complex conjugate transpose of M. Furthermore, M H denotes ( M H) 1. The real and imaginary components of the matrix M are given by Re M and ImM respectively. Finally, the set of generalized eigenvalues for the matrix pair (A, B), where A C n n and B C n n, is denoted by sp(a, B); i.e. Ax = λbx for λ sp(a, B) and for some non-zero x C n. x is a generalized eigenvector of (A, B). The set of real generalized eigenvalues of (A, B) is denoted

3 by sp r (A, B), where sp r (A, B) sp(a, B). 3.1 Real Perturbation Values The following definition is made in [1]. Definition 3.1. Given M C q l, the i-th real perturbation values of the first kind of M are defined as: τ i (M) := 1 inf{ R l q, dimker(i l M) i} (2) and the real perturbation values of the second kind are defined as: where i N. τ i (M) := inf { R q l, rank(m ) < i } (3) The following result from [1] provides two formulas for computing both kinds of real perturbation values. Theorem 3.1 ([1]). Given M C q l and i N, τ i (M) = inf γ (0,1] σ 2i(P(γ, M)) (4) τ i (M) = sup σ 2i 1 (P(γ, M)) (5) γ (0,1] where [ P(γ, M) := Re M γ 1 Im M γ Im M Re M ] (6) Remark 3.1. The function to be minimized in (4) is quasiconvex for i = 1 (see [8]), while the function to be maximized in (5) is quasiconcave for i = min(q, l) (see [4]). Hence, for these two particular cases, the corresponding real perturbation values can be computed efficiently using, for example, golden-section search methods. For the general case, however, the functions to be minimized in (4), or maximized in (5), may have multiple local extrema. Hence, we cannot use such unimodal techniques in general. The algorithm proposed in this paper efficiently computes the real perturbation values of a matrix for the general case. Remark 3.2. Note that if M is a real matrix, then the real perturbation values of M are the same as the singular values of M. To avoid talking about this trivial case, we will assume in the rest of the paper that M is not a purely real matrix; i.e. M has at least one complex element.

4 4 Preliminary Results Before presenting the proposed algorithm for evaluating (4) and (5), the following theorem is first presented, which is used in the development of the algorithm. Theorem 1. Given M C q l and real x > 0, then, ([ ]) Re M γ Im M x σ γ 1 γ2 1 Im M Re M γ 2 sp(a(x, M),B(x, M)) (7) + 1 where σ( ) denotes the set of singular values of ( ) and [ ] 0 M A(x, M) := H M x 2 I M T M x 2 I 0 [ ] M B(x, M) := H M x 2 I 0 0 M T M x 2 I Proof. Let α = γ2 1 γ 2 +1 where for n N,. It can easily be shown (e.g. see [5]) that [ M 0 P(γ, M) = Tα,q 1 0 M [ T α,n := α I n ] T α,l (8) i 1 α I n 1 1+α I n i 1 α I n ] (9) ([ I αi and satisfies T α,n Tα,n H = αi I the following: ]) 1. The rest of the proof then follows from ([ ]) 0 P(γ, M) x σ(p(γ, M)) x sp P(γ, M) H 0 ([ ]) xi 2q P(γ, M) det P(γ, M) H = 0 xi 2l [ ] M 0 xt α,q Tα,q H det [ ] 0 M M H 0 0 M T xt H α,l T 1 α,l det ( xt α,q Tα,q H ) det ( xt H α,l T 1 α,l [ M H 0 ( det x = 0 ] [ ]) ( xtα,q 0 M T Tα,q H ) 1 M 0 = 0 0 M [ ] I αi + 1 [ ]) M H M αm H M αi I x αm T M M T = 0 M det(a(x, M) αb(x, M)) = 0

5 Lemma 2. The generalized eigenvalues of (A(x, M), B(x, M)) in (7) are symmetrical both about the imaginary axis and about the real axis. Proof. [ Let α ] C be a generalized eigenvalue of (A(x, M),B(x, M)), and define 0 I J 1 :=. Then, I 0 det(a(x, M) αb(x, M)) = 0 det ( J 1 (A(x, M) αb(x, M))J1 1 ) = 0 ([ ] [ 0 M det T M x 2 I M M H M x 2 α T M x 2 I 0 I 0 0 M H M x 2 I det(a(x, M) + αb(x, M)) = 0 ]) = 0 Hence, [ α is also ] a generalized eigenvalue of (A(x, M), B(x, M)). Similarly, define 0 I J 2 := and note that I 0 det(a(x, M) αb(x, M)) = 0 det(j 2 (A(x, M) αb(x, M)) J 2 ) = 0 Hence, α is also a generalized eigenvalue of (A(x, M),B(x, M)). Lemma 3. ([2]) Let A(γ) C n n and B(γ) C n n both be analytic symmetric matrix functions on an open set Γ R. Also, let (λ(γ),y(γ)) be a generalized eigenvalue and eigenvector pair such that A(γ) y(γ) = λ(γ) B(γ) y(γ) y(γ) T B(γ)y(γ) = 1 (10a) (10b) Then, dλ(γ) = y(γ) T ( A(γ) ) B(γ) λ(γ) y(γ) (11) Lemma 4. dw(γ) Given M C q l and w(γ) σ(p(γ, M)), then, ( ) γ 2γ 2 1 = w(γ)(γ 2 + 1) 2 y(γ) T γ 2 +1 I I ) y(γ) I I ( γ 2 1 γ 2 +1 ( ) where γ 2 1 γ 2 +1, y(γ) is a generalized eigenvalue and eigenvector pair of (A(w(γ),M), B(w(γ),M)), as defined in (7), such that A(w(γ),M)y(γ) = γ2 1 γ B(w(γ),M)y(γ) y(γ) T B(x, M)y(γ) = 1 1 (12)

6 Proof. Let α(γ) = γ2 1 γ Since w(γ) is a singular value of P(γ, M), then by Theorem 7, α(γ) is a generalized eigenvalue of (A(w(γ),M),B(w(γ),M)). Let y(γ) be a corresponding generalized eigenvector such that y(γ) T B(w(γ),M)y(γ) = 1 (such a y(γ) can always be obtained by the appropriate scaling of any eigenvector corresponding to α(γ)). Then by Lemma 3, ( dα(γ) da(w(γ),m) dw(γ) = y(γ)t α(γ) db(w(γ),m) ) y(γ) dw(γ) dw(γ) [ Noting that da(w(γ),m) 0 I dw(γ) = 2w(γ) I 0 then the proof follows from dw(γ) = dα(γ) ] dw(γ) dα(γ). [ and db(w(γ),m) I 0 dw(γ) = 2w(γ) 0 I ] 5 Algorithm An algorithm for computing the i-th real perturbation value (i.e. solving (4) and (5)) of a given M C q l, for i N, is now presented in this section. The following development focuses mainly on computing the real perturbation values of the second kind (i.e solving (5)). The algorithm to compute the real perturbation values of the first kind (i.e. to solve (4)) is similar. Firstly, let the global maximum of (5) be denoted by: ([ ]) r Re M γ Im M = sup σ 2i 1 γ (0,1] γ 1 Im M Re M with the global maximizer being γ (0, 1]; i.e. r = σ 2i 1 (P(γ, M)). Now suppose that at the k-th iteration, for k = 1, 2,..., we are given r k 1 and γ k 1 (i.e. obtained from the previous iteration), which are approximations of r and γ respectively, where r k 1 = σ 2i 1 (P(γ k 1, M)). The basic concept of the algorithm is then to use the information of r k 1 and γ k 1, along with Theorem 1 and Lemma 4, to obtain the so-called maximizing set (see [9]), S k, defined as: S k = {γ (0, 1] σ 2i 1 (P(γ, M)) > r k 1 } The global maximizer γ is therefore contained in S k. We then select a new point γ k S k, which achieves r k > r k 1, where r k = σ 2i 1 (P(γ k, M)). r k is then a new (and better) approximation of the global maximum r. The procedure is then repeated until the size of S k is smaller than an user-specified tolerance. 5.1 Obtaining the maximizing set, S k Let the current approximation of r and γ be r k 1 and γ k 1 respectively. Due to continuity, S k is a set of intervals in (0, 1], where the endpoints of these intervals are

7 all γ (0, 1] such that σ 2i 1 (P(γ, M)) = r k 1, and also possibly 0 and 1. Hence, S k is obtained by finding these endpoints. By Theorem 1, all γ (0, 1] that achieve a singular value of P(γ, M) equal to r k 1 (i.e. σ t (P(γ, M)) = r k 1 for some t N) can be obtained from the set of real generalized eigenvalues of (A(r k 1, M),B(r k 1, M)) as defined in (7). In particular, let Γ(r k 1 ) := {γ (0, 1] t N, σ t (P(γ, M)) = r k 1 } (13) { = γ (0, 1] γ 2 } 1 γ Λ r (by Theorem 1 and fact that γ is real) where we denote Λ r := sp r (A(r k 1, M),B(r k 1, M)). To determine which singular value of P(γ, M) equals r k 1 (i.e. to determine t such that σ t (P(γ, M)) = r k 1 ) for γ Γ(r k 1 ), one can compute the singular values of P(γ, M) and compare them with r k 1 for all γ Γ(r k 1 ). However, by Lemma 2, there can be at most min(q, l) elements in Γ(r k 1 ) for a given M C q l. Hence, computing the singular values of P(γ, M) for all γ Γ(r k 1 ) can be relatively costly. Instead, we use the derivative information at each γ Γ(r k 1 ) to determine which singular value of P(γ, M) equals r k 1. The procedure is described as follows, which is similar to a technique found in [9]. Let Γ(r k 1 ) be sorted in ascending order; i.e. Γ(r k 1 ) = {g 1, g 2,...} such that g 1 g 2. Also, let g w = γ k 1 for some w N (note that w always exists since r k 1 = σ 2i 1 (P(γ k 1, M)), which implies γ k 1 Γ(r k 1 )). Suppose that the derivative (as computed by (12) of Lemma 4) of the singular value of P(g w, M), denoted by dσ(p(gw,m)), is positive. Then, this implies that the plot of the (2i 1)-th singular value of P(γ, M) as a function of γ (0, 1], denoted y 2i 1 (γ) = σ 2i 1 (P(γ, M)), crosses upward at g w. Next, consider the derivative at g w+1 ; i.e. dσ(p(gw+1,m)). If the derivative is again positive, then this means that there is another upward crossing at g w+1. Such an upward crossing can only occur from the plot y 2i 2 (γ) = σ 2i 2 (P(γ, M)). This is because if one considers all the singular value plots y t (γ) = σ t (P(γ, M)) for t = 1,...,2 min(q, l), then by the ordering of the singular values, we have y t1 (γ) y t2 (γ) for all γ (0, 1] and for all t 1 t 2. In other words, the plot y t1 (γ) is always above (or equal to) the plot y t2 (γ) for all t 1 t 2. Hence, an upward crossing at g w+1 can only be by the next plot below y 2i 1 (γ); i.e. y 2i 2 (γ). On the other hand, if the derivative at g w+1 is negative, then there is a downward crossing at g w+1, which can only occur from y 2i 1 (γ) for a similar reason. Hence, starting from g w and working in the manner as described above, the upward and downward crossings are used to determine which singular value plot crosses at each g Γ(r k 1 ). Figure 1 illustrates a possible example scenario, where the + and signs depict an upward and downward crossing respectively. To obtain the maximizing set, S k, we are mainly interested in determining all γ Γ(r k 1 ) such that σ 2i 1 (P(γ, M)) = r k 1 ; i.e. all g Γ(r k 1 ) where there is a crossing by the y 2i 1 (γ) plot. Define such a set as Γ(r k 1 ) := {γ Γ(r k 1 ) σ 2i 1 (P(γ, M)) = r k 1 } (14)

8 2i 2i 1 2i 1 2i 2 2i 2 2i g w 2 g w 1 g w+1 g w+2 g w+3 g w Figure 1. Example of using the derivative information of g Γ(r k 1 ) to determine the maximizing set, S k 1. The maximizing set, S k, is hence bounded by the elements of Γ(r k 1 ) (and also possibly by 0 and 1); i.e. Γ(r k 1 ) contains the endpoints of the interval(s) in (0, 1] that achieves σ 2i 1 (P(γ, M)) greater than r k 1. The derivative information obtained previously can again be used to determine which elements in Γ(r k 1 ) are at the beginning of an interval and which are at the end. In particular, since we are searching for a maximum, all beginning endpoints will thus have an upward crossing and all ending endpoints will have a downward crossing. Referring back to the example scenario shown in Figure 1, the maximizing set is hence S k = (0, g w 1 ) (g w, g w+3 ) (assuming there are no more upward σ 2i 1 crossings in (0, g w 2 ) and (g w+3, 1]). 5.2 Updating the current maximum, r k By the construction of S k, any γ S k achieves σ 2i 1 (P(γ, M)) greater than r k 1. Hence, the next approximation of the global maximizer, γ k, can arbitrarily be chosen in S k, and then set r k = σ 2i 1 (P(γ k, M)) > r k 1. From experience, however, the performance of the algorithm seems to improve by choosing γ k as the maximizer of a cubic fit through the endpoints (and their derivatives) of the intervals of S k (see [9] and pseudo-code below). 5.3 Algorithm outline The algorithm can be summarized as follows: Algorithm 5.1 Computing the real perturbation value (of the second kind) of M Input: M C q l and i N Input tolerance: TOL Sk Output: r and γ, where r = σ 2i 1 (P(γ, M)) 1. Initialization: Choose arbitrary γ 0 (0, 1] and compute r 0 = σ 2i 1 (P(γ 0, M)). 2. Iteration k(= 1, 2,...) (a) Given r k 1 and γ k 1.

9 (b) Compute Λ r = sp r (A(r k 1, M),B(r k 1, M)). (c) Obtain the set (sorted in ascending order) { Γ(r k 1 ) = γ (0, 1] γ 2 } 1 γ Λ r (d) For each g Γ(r k 1 ), compute the derivative dσ(p(g,m)) via (12). (e) Using dσ(p(g,m)) for each g Γ(r k 1 ), determine Γ(r k 1 ) := {γ Γ(r k 1 ) σ 2i 1 (P(γ, M)) = r k 1 } (f) Find the maximizing set S k = {γ (0, 1] σ 2i 1 (P(γ, M)) > r k 1 } via Γ(r k 1 ) and dσ(p(g,m)) for each g Γ(r k 1 ). (g) Quit if the largest interval of S k is less than TOL Sk. (h) For each interval s j S k, Find cubic fit through the endpoints (and their derivatives) of s j. Define m j := maximizer of cubic fit within s j. Evaluate σ 2i 1 (P(m j, M)). Set γ k to whichever m j achieves a larger σ 2i 1 (P(m j, M)). 3. Update k k + 1 and repeat step Computational requirements To provide an idea of the computational requirements of using Algorithm 5.1 for computing the real perturbation values, the number of operations required in terms of the number of singular value (σ) and eigenvalue (λ) problems are noted in Table 1. As it can be seen from Table 1, only a small number of eigenvalue and singular value problems are solved at each iteration. In particular, only one generalized eigenvalue problem of size 2 min(q, l) is solved at step (2b). Then in step (2h), at least one singular value problem of size 2q 2l is solved in order to obtain the next approximation of the real perturbation value. However, since Γ(r k 1 ) in (2c) contains at most min(q, l) elements (by Lemma 2), then the maximizing set contains at most min(q,l) 2 min(q,l) intervals. Therefore, step (2h) requires at most + 1 singular value problems to be solved. Furthermore, it should be noted that the derivative information at step (2d) is computed at relatively low cost. As shown in (12), the derivatives are functions of the generalized eigenvectors of (A(r k 1, M),B(r k 1, M)) as defined in (7). Since these eigenvectors are already known from the eigendecomposition of (A(r k 1, M),B(r k 1, M)) in step (2b), the derivatives can be obtained very easily.

10 Table 1. Summary of operation count of Algorithm 5.1 Step # of (σ,λ) Size of matrix 1 1 σ 2q 2l 2(b) 1 λ 2(min(q, l)) 2(min(q, l)) 2(h) at least 1 σ 2q 2l at most ( min(q,l) ) σ Here, σ and λ denote the number of singular value and eigenvalue problem(s). 6 Numerical Example Consider the following matrix M = i i i where we are interested in computing τ 3 (M). Using a fine grid search, the third real perturbation value of M is found to be τ 3 (M) = , achieved at γ = Using the proposed algorithm with the (arbitrary) starting point γ 0 = 0.5, τ 3 (M) is found to within 3 significant figures in 1 iteration and to 9 significant figures in 2 iterations. Table 2 lists r k, γ k, and the maximizing set, S k, for each iteration k = 0,...,4. Figure 2 plots the maximizing set, S k, obtained at Iterations 1 and 2. It can be seen that in this example, the maximizing set is significantly reduced by the second iteration. Furthermore, the proposed algorithm only required solving a total of 4 generalized eigenvalue and 10 singular value problems for this example, which is much less demanding than performing a fine grid search. Table 2. Estimates of the global maximum (r k ), the maximizer (γ k ), and the maximizing set (S k ) at each iteration k Iteration (k) γ k r k S k ( , ) ( , ) ( , ) ( , ) ( , ) ( , )

11 Iteration σ 2i 1 (P(γ,M)) γ Iteration 2 σ 2i 1 (P(γ,M)) cubic fit maximizing set, S k σ 2i 1 (P(γ,M)) cubic fit maximizing set, S k 1.1 σ 2i 1 (P(γ,M)) Figure 2. Intermediate results at Iteration 1 (top) and Iteration 2 (bottom). 7 Conclusions In this paper, an efficient algorithm is presented for computing the real perturbation values of a given complex matrix. The algorithm discussed here focuses mainly on solving the real perturbation values of the second kind (i.e. solving the optimization in (5)). Modifying the algorithm to solve the real perturbation values of the first kind (4) is straightforward and is left for the reader. γ

12 Bibliography [1] B. Bernhardsson, A. Rantzer, and L. Qiu. Real perturbation values and real quadratic forms in a complex vector space. Linear Algebra and its Applications, 270: , [2] J. de Leeuw. Derivatives of Generalized Eigen Systems with Applications. Department of Statistics, UCLA, January Paper [3] R. Eising. Between controllable and uncontrollable. Systems and Control Letters, 4: , [4] G. Hu and E. J. Davison. Real controllability/stabilizability radius of LTI systems. IEEE Trans. Automat. Contr., 49(2): , [5] M. Karow. Geometry of Spectral Value Sets. PhD thesis, University of Bremen, Germany, [6] S. Lam and E. J. Davison. The real decentralized fixed mode radius of LTI systems. In 46 th IEEE Conference on Decision and Control, pages , New Orleans, LA, USA, December [7] S. Lam and E. J. Davison. The transmission zero at s radius and the minimum phase radius of LTI systems. In 17 th IFAC world congress, Seoul, Korea, July To appear. [8] L. Qiu, B. Bernhardsson, A. Rantzer, E. J. Davison, P. M. Young, and J. C. Doyle. A formula for computation of the real stability radius. Automatica, 31(6): , [9] J. Sreedhar, P. Van Dooren, and A. L. Tits. A fast algorithm to compute the real structured stability radius. In International Series of Numerical Mathematics, volume 121, pages Birkhäuser Verlag Basel, 1996.

A fast algorithm to compute the controllability, decentralized fixed-mode, and minimum-phase radius of LTI systems

Proceedings of the 7th IEEE Conference on Decision Control Cancun, Mexico, Dec. 9-, 8 TuA6. A fast algorithm to compute the controllability, decentralized fixed-mode, minimum-phase radius of LTI systems