Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation Thanos Antoulas Rice University and Jacobs University email: aca@rice.edu URL: www.ece.rice.edu/ aca International School, Monopoli, 7-12 September 2008 Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 1 / 38
Outline 1 Krylov approximation methods 2 The Arnoldi and the Lanczos procedures The Arnoldi procedure The Lanczos procedure An example 3 Krylov methods and moment matching Remarks 4 Rational interpolation by Krylov projection Realization by projection Interpolation by projection 5 Choice of Krylov projection points: Optimal H 2 model reduction 6 Summary: Lectures II and III Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 2 / 38
Outline Krylov approximation methods 1 Krylov approximation methods 2 The Arnoldi and the Lanczos procedures The Arnoldi procedure The Lanczos procedure An example 3 Krylov methods and moment matching Remarks 4 Rational interpolation by Krylov projection Realization by projection Interpolation by projection 5 Choice of Krylov projection points: Optimal H 2 model reduction 6 Summary: Lectures II and III Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 3 / 38
Krylov approximation methods Krylov approximation methods ( A B Given Σ = C D ), expand the transfer function around s 0 : H(s) = η 0 + η 1 (s s 0 ) + η 2 (s s 0 ) 2 + η 3 (s s 0 ) 3 + ( ) Â ˆB Moments at s 0 : η j, j 0. Find ˆΣ = Ĉ ˆD, with Ĥ(s) = ˆη 0 + ˆη 1 (s s 0 ) + ˆη 2 (s s 0 ) 2 + ˆη 3 (s s 0 ) 3 + such that for appropriate k: η j = ˆη j, j = 1, 2,, k Moment matching methods can be implemented in a numerically stable and efficient way. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 4 / 38
Krylov approximation methods Krylov approximation methods: Special cases s 0 = Moments: Markov parameters Problem: (partial) realization Solution computed through: Lanczos and Arnoldi procedures s 0 = 0 Problem: Padé approximation Solution computed through: Lanczos and Arnoldi procedures In general: arbirtary s 0 C Problem: Rational interpolation Solution computed through: Rational Lanczos Computation of moments: numerically problematic Key fact for numerical reliability: If (A, B, C, D) given moment matching without moment computation iterative implementation. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 5 / 38
Outline The Arnoldi and the Lanczos procedures 1 Krylov approximation methods 2 The Arnoldi and the Lanczos procedures The Arnoldi procedure The Lanczos procedure An example 3 Krylov methods and moment matching Remarks 4 Rational interpolation by Krylov projection Realization by projection Interpolation by projection 5 Choice of Krylov projection points: Optimal H 2 model reduction 6 Summary: Lectures II and III Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 6 / 38
The Arnoldi and the Lanczos procedures The Arnoldi procedure The Arnoldi procedure Given is A R n n, and b R n. Let R k (A, b) R n k be the reachability or Krylov matrix. It is assumed that R k has full column rank equal to k. Devise a process which is iterative and at the k th step we have AV k = V k H k + R k, V k, R k R n k, H k R k k, k = 1, 2,, n H A V = V + R These quantities have to satisfy the following conditions at each step. The columns of V k are orthonormal: V k V k = I k, k = 1, 2,, n. span col V k = span col R k (A, b), k = 1, 2,, n The residual R k satisfies the Galerkin condition: V k R k = 0, k = 1, 2,, n. This problem can be solved by the Arnoldi procedure. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 7 / 38
The Arnoldi and the Lanczos procedures The Arnoldi procedure Arnoldi: recursive implementation Given: A R n n, b R n Find: V R n k, f R n, and H R k k, such that AV = VH + fe k where H = V AV, V V = I k, V f = 0, with H in upper Hessenberg form. 1 v 1 = b b, w = Av 1; α 1 = v 1 w f 1 = w v 1 α 1 ; V 1 = (v 1 ); H 1 = (α 1 ) 2 For j = 1, 2,, k 1 1 β j = f j, v j+1 = f j β j ( Hj 2 V j+1 = ( ) V j v j+1, Ĥ j = β j e j 3 w = Av j+1, h = V j+1 w, f j+1 = w V j+1 h ) 4 H j+1 = (Ĥj h ) Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 8 / 38
The Arnoldi and the Lanczos procedures The Arnoldi procedure Properties of Arnoldi H k is obtained by projecting A onto the span of the columns of V k : H k = V k AV k. The remainder R k has rank one and can be written as R k = r k e k, where e k is the kth unit vector; thus r k R k. This further implies that v k+1 = Hessenberg matrix. H k = r k r k, where v k+1 is the (k + 1)st column of V. Consequently, H k is an upper h 1,1 h 1,2 h 1,3 h 1,k 1 h 1,k h 2,1 h 2,2 h 2,3 h 2,k 1 h 2,k h 3,2 h 3,3 h 3,k 1 h 3,k......... h k 1,k 1 h k,k 1 h k 1,k h k,k Let p k (λ) = det(λi k H k ), be the characteristic polynomial of H k. This monic polynomial is the solution of the following minimization problem p k = arg min p(a)b 2 where the minimum is taken over all monic polynomials p of degree k. Since p k (A)b = A k b + R k p, where p i+1 is the coefficient of λ i of the polynomial p k, it also follows that the coefficients of p k provide the least squares fit between A k b and the columns of R k. There holds r k = 1 p k 1 (A)b p k (A)b, H k,k 1 = p k (A)b p k 1 (A)b Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 9 / 38
The Arnoldi and the Lanczos procedures The Arnoldi procedure An alternative way of looking at Arnoldi Consider a matrix A R n n, a starting vector b R n, and the corresponding reachability matrix R n = [b Ab A n 1 b]. The following relationship holds true: 0 0 0 α 0 1 0 0 α 1 AR n = R n F where F = 0 1 0 α 2. 0 0 1 α n 1 and χ A (s) = s n + α n 1 s n 1 + + α 1 s + α 0, is the characteristic polynomial of A. Compute the QR factorization of R n : R n = VU, V V = I n, U upper triangular Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 10 / 38
The Arnoldi and the Lanczos procedures The Arnoldi procedure It follows that AVU = VUF AV = V } UFU {{ 1 } Ā AV = VĀ Since U is upper triangular, so is U 1 ; furthermore F is upper Hessenberg. Therefore Ā being the product of an upper triangular times an upper Hessenberg times an upper triangular matrix is upper Hessenberg. The k-step Arnoldi factorization can now be obtained by considering the first k columns of the above relationship, to wit: [AV] k = [ VĀ] k A[V] k = [V] k Ā kk + fe k where f is a multiple of the (k +1)-st column of V. Notice that Ākk is still upper Hessenberg, while the columns of [V] k provide an orthonormal basis for the space spanned by the first k columns of the reachability matrix R n. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 11 / 38
The Arnoldi and the Lanczos procedures The Lanczos procedure The symmetric Lanczos procedure If A = A then the Arnoldi procedure is the same as the symmetric Lanczos procedure. In this case H k is tridiagonal: H k = α 1 β 2 β 2 α 2 β 3 β 3 α 3......... α k 1 β k This matrix shows that the vectors in the Lanczos procedure satisfy a three term recurrence relationship β k α k Av i = β i+1 v i+1 + α i v i + β i v i 1, i = 1, 2,, k 1 Remark. If the remainder r k = 0, the procedure has terminated, in which case if (λ, x) is an eigenpair of H k, (λ, V k x) is an eigenpair of A (since H k x = λx implies AV k x = V k H k x = λv k x). Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 12 / 38
The Arnoldi and the Lanczos procedures The Lanczos procedure Two-sided Lanczos The two-sided Lanczos procedure. Given A R n n which is not symmetric, and two vectors b, c R n, devise a process which is iterative and the k th step there holds: AV k = V k H k + R k, A W k = W k H k + S k, k = 1, 2,, n. Biorthogonality: W k V k = I k, span col V k = span col R k (A, b), span col W k = span col R k (A, c ), Galerkin conditions: V k S k = 0, W k R k = 0, k = 1, 2,, n. Remarks. The second condition of the second item above can also be expressed as span rows W k = span rows O k (c, A), where O k is the observability matrix of the pair (c, A). The assumption for the solvability of this problem is det O k (c, A)R k (A, b) 0, k = 1, 2,, n. The associated Lanczos polynomials are defined as p k (λ) = det(λi k H k ), and the induced inner product is defined as p(λ), q(λ) = p(a )c, q(a)b = c p(a) q(a) b. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 13 / 38
The Arnoldi and the Lanczos procedures The Lanczos procedure Two-sided Lanczos: recursive implementation Given: the triple A R n n, b, c R n Find: V, W R n k,, g R n, and H R k k, such that AV = VH + fe k, A W = WH + ge k where H = V AW, V W = I k, W f = 0, V g = 0. The projections π L and π U above, are given by V, W, respectively. 1 β 1 := b c, γ 1 := sgn (b c )β 1 v 1 = b/β 1, w 1 := c /γ 1 2 For j = 1,, k, set 1 α j = w j Av j 2 r j = Av j α j v j γ j v j 1, q j = A w j α j w j β j w j 1 3 β j+1 = r j q j, γ j+1 = sgn (r j q j)β j+1 4 v j+1 = r j /β j+1, w j+1 = q j /γ j+1 Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 14 / 38
The Arnoldi and the Lanczos procedures The Lanczos procedure Properties of two-sided Lanczos H k is obtained by projecting A as follows: H k = W k AV k. The remainders R k, S k have rank one and can be written as R k = r k e k, S k = q k e k. This further implies that v k+1, w k+1 are scaled versions of r k, q k respectively Consequently, H k is a tridiagonal matrix. The generalized Lanczos polynomials p k (λ) = det(λi k H k ), k = 0, 1,, n 1, p 0 = 1, are orthogonal: p i, p j = 0, for i j. The columns of V k, W k and the Lanczos polynomials satisfy the following three-term recurrences γ k v k+1 = (A α k )v k β k 1 v k 1 β k w k+1 = (A α k )w k γ k 1 w k 1 γ k p k+1 (λ) = (λ α k )p k (λ) β k 1 p k 1 (λ) β k q k+1 (λ) = (λ α k )q k (λ) γ k 1 q k 1 (λ) Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 15 / 38
The Arnoldi and the Lanczos procedures An example Example: symmetric Lanczos Consider the following symmetric matrix: A = 2 1 2 1 1 2 0 1 2 0 2 1 1 1 1 0 With the starting vector b = [1 0 0 0], we obtain Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 16 / 38
The Arnoldi and the Lanczos procedures V 2 = V 3 = V 4 = where V 1 = 1 0 0 1 6 0 2 6 0 1 6 1 0 0 0 6 1 3 1 0 2 6 1 3 0 1 6 1 3 1 0 0 0 0 6 1 3 1 2 1 0 2 6 0 1 6 1 3 1 3 0 [ 1 0 0 0 An example ] [ 0 1, H 1 = [2], R 1 = 2 1 [, H 2 =, H 3 = 1 2 ] 2 6 6 8 3 [ 2 6 0 6 8 3 1 18 0 1 18 4 3, H 4 = ], R 2 = 0 0 0 1 54 0 ], R 3 = 2 6 0 0 6 8 1 3 0 18 0 1 4 3 18 3 2 0 0 3 0 2 1 54 0 1 54 [ 0 0 0 0 0 3 2 0 0 0 3 0 0 2 ], R 4 = 0 4 4 AV k = V k H k + R k, V k R k = 0 H k = V k AV k, k = 1, 2, 3, 4. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 17 / 38
Outline Krylov methods and moment matching 1 Krylov approximation methods 2 The Arnoldi and the Lanczos procedures The Arnoldi procedure The Lanczos procedure An example 3 Krylov methods and moment matching Remarks 4 Rational interpolation by Krylov projection Realization by projection Interpolation by projection 5 Choice of Krylov projection points: Optimal H 2 model reduction 6 Summary: Lectures II and III Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 18 / 38
Krylov methods and moment matching Arnoldi and moment matching The Arnoldi factorization can be used for model reduction as follows. Recall the QR factorization of the reachability matrix R k R n k ; a projection VV can then be attached to this factorization: R k = VU V = R k U 1 where V R n k, V V = I k, and U is upper triangular. The reduced order system is: Σ = ( Ā B C ) where Ā = V AV, B = V B, C = CV Theorem. Σ as defined above satisfies the equality of the Markov parameters ˆη i = η i, i = 1,, k. Furthermore, Ā is in Hessenberg form, and B is a multiple of the unit vector e 1. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 19 / 38
Krylov methods and moment matching Proof. First notice that since U is upper triangular, v 1 = B B, and since V R k = U it follows that B = u 1 = B e 1 ; therefore B = V B. VV B = V B = B, hence Ā B = V AVV B = V AB; in general, since VV is a projection along the columns of R k, we have VV R k = R k ; moreover: ˆR k = V R k ; hence (ˆη 1 ˆη k ) = Ĉ ˆR k = CVV R k = CR k = (η 1 η k ) Finally, the upper triangularity of U implies that A is in Hessenberg form. Remark. Similarly, one can show that reduction by means the two-sided Lanczos procedure preserves 2k Markov parameters. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 20 / 38
Krylov methods and moment matching Remarks Remarks The number of operations is O(k 2 n) vs. O(n 3 ), which implies efficiency. The requirement for memory is large if k is relatively large. Only matrix-vector multiplications are required. No matrix factorizations and/or inversions. There is no need to compute the transformed n-th order model and then truncate. This eliminates ill-conditioning. Drawbacks: Numerical issue: Arnoldi/Lanczos methods loose orthogonality. This comes from the instability of the classical Gram-Schmidt procedure. Remedy: re-orthogonalization. no global error bound. ˆΣ tends to approximate the high frequency poles of Σ. Remedy: match expansions around other frequencies rational Lanczos. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 21 / 38
Outline Rational interpolation by Krylov projection 1 Krylov approximation methods 2 The Arnoldi and the Lanczos procedures The Arnoldi procedure The Lanczos procedure An example 3 Krylov methods and moment matching Remarks 4 Rational interpolation by Krylov projection Realization by projection Interpolation by projection 5 Choice of Krylov projection points: Optimal H 2 model reduction 6 Summary: Lectures II and III Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 22 / 38
Rational interpolation by Krylov projection Realization by projection Partial realization by projection Given a system Σ = (A, B, C), where A R n n and B, C R n, We seek a lower dimensional model ˆΣ = (Â, ˆB, Ĉ), where  Rk k, ˆB, Ĉ R k, k < n, such that ˆΣ preserves some properties of the original system, through appropriate projection methods. In other words, we seek V R n k and W R n k such that W V = I k, and the reduced system is given by:  = W AV, ˆB = W B, Ĉ = CV. Lemma With V = [B, AB,, A k 1 B] = R k (A, B) and W any left inverse of V, ˆΣ is a partial realization of Σ and matches k Markov parameters. From a numerical point of view, one would not use V as defined above since usually the columns of V are almost linearly dependent. As it turns out any matrix whose column span is the same as that of V can be used. Proof. We have ĈˆB = CVW B = CR k (A, B)e 1 = CB; furthermore ĈÂj ˆB = CR k (A, B)W A j R k (A, B)e 1 = CR k (A, B)W A j B = CR k (A, B)e j+1 = CA j B, j = 1,, k 1. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 23 / 38
Rational interpolation by Krylov projection Interpolation by projection Rational interpolation by projection Suppose now that we are given k distinct points s j C. V is defined as the generalized reachability matrix V = [ (s 1 I n A) 1 B,, (s k I n A) 1 B ], and as before, let W be any left inverse of V. Then Lemma ˆΣ defined above, interpolates the transfer function of Σ at the s j, that is H(s j ) = C(s j I n A) 1 B = Ĉ(s ji k Â) 1 ˆB = Ĥ(s j ), j = 1,, k. Proof. The following string of equalities leads to the desired result: Ĉ(s j I k Â) 1 ˆB = CV(sj I k W AV) 1 W B [ = C (s 1 I n A) 1 B,, (s k I n A) 1 ] ( B W 1 (s j I n A)V) W B [ = C(s 1 I n A) 1 B,, C(s k I n A) 1 ) 1 B] ([ W B ] W B [ = C(s 1 I n A) 1 B,, C(s k I n A) 1 ] B e j = C(s j I n A) 1 B. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 24 / 38
Rational interpolation by Krylov projection Interpolation by projection Matching points with multiplicity We now wish to match the value of the transfer function at a given point s 0 C, together with k 1 derivatives. For this we define the generalized reachability matrix V = together with any left inverse W thereof. Lemma [ ] (s 0 I n A) 1 B, (s 0 I n A) 2 B,, (s 0 I n A) k B, ˆΣ interpolates the transfer function of Σ at s 0, together with k 1 derivatives at the same point, j = 0, 1,, k 1: ( 1) j d j j! ds j H(s) = C(s 0 I n A) (j+1) B = Ĉ(s ( 1) 0I k Â) (j+1) ˆB j d j = s=s0 j! ds j Ĥ(s) s=s0 Proof. Let V be as defined as above, and W be such that W V = I k. It readily follows that the projected matrix s 0 I r  is in companion form (expression on the left) and therefore its powers are obtained by shifting its columns to the right: s 0 I k  = W (s 0 I n A)V = [W B, e 1,, e k 1 ] (s 0 I k Â)l = [, W B, e }{{} 1,, e k l ]. l 1 Consequently [W (s 0 I n A)V] l W B = e l, which finally implies Ĉ(s 0 I k Â) l ˆB = CV [W (s 0 I A)V] l W B = CVe l = C(s 0 I n A) l B, l = 1, 2,, k. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 25 / 38
Rational interpolation by Krylov projection Interpolation by projection General result: rational Krylov A projector which is composed of any combination of the above three cases achieves matching of an appropriate number of Markov parameters and moments. Let the partial reachability matrix be R k (A, B) = [ B AB A k 1 B ], and partial generalized reachability matrix be: R k (A, B; σ) = [ (σi n A) 1 B (σi n A) 2 B (σi n A) k B ]. Rational Krylov (a) If V as defined in the above three cases is replaced by V = VR, R R k k, det R 0, and W by W = R 1 W, the same matching results hold true. (b) Let V be such that span col V = span col [R k (A, B) R m1 (A, B; σ 1 ) R ml (A, B; σ l )], and W any left inverse of V. The reduced system matches k Markov parameters and m i moments at σ i C, i = 1,, l. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 26 / 38
Rational interpolation by Krylov projection Interpolation by projection Two-sided projections: the choice of W Let O k (C, A) R k n, be the partial observability matrix consisting of the first k rows of O n (C, A) R n n. The first case is V = R k (A, B), W = (O k (C, A)R k (A, B) ) 1 O k (C, A). }{{} H k Lemma Assuming that det H k 0, ˆΣ is a partial realization of Σ and matches 2k Markov parameters. Given 2k distinct points s 1,, s 2k, we will make use of the following generalized reachability and observability matrices: Ṽ = Lemma [ ] (s 1 I n A) 1 B (s k I n A) 1 B, W = [(s k+1 I n A ) 1 C (s 2k I n A ) 1 C ]. Assuming that det W Ṽ 0, the projected system ˆΣ where V = Ṽ and W = W(Ṽ W) 1 interpolates the transfer function of Σ at the 2k points s i. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 27 / 38
Rational interpolation by Krylov projection Interpolation by projection Remarks (a)the same procedure as above can be used to approximate implicit systems, i.e., systems that are given in a generalized form Eẋ(t) = Ax(t) + Bu(t), y(t) = Cx(t), where E may be singular. The reduced system is given by Ê = W EV, Â = W AV, ˆB = W B, Ĉ = CV, where C(s k+1 E A) 1 W =., V = [ (s 1 E A) 1 B (s k E A) 1 B ] C(s 2k E A) 1 (b) Sylvester equations and projectors. The solution of an appropriate Sylvester equation AX + XH + BG = 0, provides a projector that interpolates the original system C, A, B at minus the eigenvalues of H. Therefore the projectors above can be obtained by solving Sylvester equations. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 28 / 38
Choice of Krylov projection points: Optimal H 2 model reduction Outline 1 Krylov approximation methods 2 The Arnoldi and the Lanczos procedures The Arnoldi procedure The Lanczos procedure An example 3 Krylov methods and moment matching Remarks 4 Rational interpolation by Krylov projection Realization by projection Interpolation by projection 5 Choice of Krylov projection points: Optimal H 2 model reduction 6 Summary: Lectures II and III Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 29 / 38
Choice of Krylov projection points: Optimal H 2 model reduction Choice of Krylov projection points: Optimal H 2 model reduction Recall: the H 2 norm of a stable system is: Σ H2 = ( + ) 1/2 h 2 (t)dt where h(t) = Ce At B, t 0, is the impulse response of Σ. Goal: construct a Krylov projector such that Σ k = arg min deg(ˆσ) = r ˆΣ : stable Σ ˆΣ = H2 ( + (h ĥ)2 (t)dt ) 1/2 Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 30 / 38
Choice of Krylov projection points: Optimal H 2 model reduction First-order necessary optimality conditions Let (Â, ˆB, Ĉ) solve the optimal H 2 problem and let ˆλ i denote the eigenvalues of Â. The necessary conditions are H( ˆλ i ) = Ĥ( ˆλ i ) and d ds H(s) = d s= ˆλ ds Ĥ(s) i s= ˆλ i Thus the reduced system has to match the first two moments of the original system at the mirror images of the eigenvalues of Â. The H 2 norm: if H(s) = n n φ k k=1 s λ k H 2 H 2 = c k H( λ i ) Corollary. With Ĥ(s) = r ˆφ k k=1, the H s ˆλ 2 norm of the error system, is k J = H Ĥ 2 n [ ] = φ i H( λ i ) Ĥ( λ i ) + H 2 i=1 r j=1 k=1 ] ˆφ j [Ĥ( ˆλj ) H( ˆλ j ) Conclusion. The H 2 error is due to the mismatch of the transfer functions H Ĥ at the mirror images of the full-order and reduced system poles λ i, ˆλ i. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 31 / 38
Choice of Krylov projection points: Optimal H 2 model reduction An iterative algorithm Let the system obtained after the (j 1) st step be (C j 1, A j 1, B j 1 ), where A j 1 R k k, B j 1, C j 1 Rk. At the j th step the system is obtained as follows where A j = (W j V j) 1 W j AV j, B j = (W j V j) 1 W j B, C j = CV j, V j = [ (λ 1 I A) 1 B,, (λ k I A) 1 B ], W j = [ C(λ 1 I A) 1,, C(λ k I A) 1], and: λ 1,, λ k σ(a j 1 ), i.e., λ i are the eigenvalues of the (j 1) st iterate A j 1. The Newton step: can be computed explicitly λ (k) 1 λ (k) 2. local convergence guaranteed. λ (k) 1 λ (k) 2. J 1 λ (k 1) 1 λ (k 1) 2. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 32 / 38
Choice of Krylov projection points: Optimal H 2 model reduction An iterative rational Krylov algorithm (IRKA) The proposed algorithm produces a reduced order model Ĥ(s) that satisfies the interpolation-based conditions, i.e. H( ˆλ i ) = Ĥ( ˆλ d i ) and ds H(s) = d s= ˆλ ds Ĥ(s) s= ˆλ i i 1 Make an initial selection of σ i, for i = 1,..., k 2 W = [(σ 1 I A ) 1 C,, (σ k I A ) 1 C ] 3 V = [(σ 1 I A) 1 B,, (σ k I A) 1 B] 4 while (not converged) Â = (W V) 1 W AV, σ i λ i (Â) + Newton correction, i = 1,..., k W = [(σ 1 I A ) 1 C,, (σ k I A ) 1 C ] V = [(σ 1 I A) 1 B,, (σ k I A) 1 B] 5 Â = (W V) 1 W AV, ˆB = (W V) 1 W B, Ĉ = CV Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 33 / 38
Choice of Krylov projection points: Optimal H 2 model reduction Moderate-dimensional example u y R L C R C R L... L C R C R L C R C total system variables n = 902, independent variables dim = 599, reduced dimension k = 21 reduced model captures dominant modes Singular values (db) 20 22 24 26 28 30 32 34 Frequency response Spectral zero method with SADPA n=902 dim=599 k=21 Original Reduced(SZM) Imag 3 2 1 0 1 Dominant spectral zeros Theoretical and found with SADPA Spz: original Spz: dominant Spz: SADPA computed 36 38 2 40 2 1.5 1 0.5 0 0.5 1 1.5 2 2.5 3 Frequency x 10 8 (rad/s) 3 0.0136 0.0135 0.0135 0.0134 0.0134 Real Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 34 / 38
Choice of Krylov projection points: Optimal H 2 model reduction H and H 2 error norms Relative norms of the error systems Reduction Method n = 902, dim = 599, k = 21 H H 2 PRIMA 1.4775 - Spectral Zero Method with SADPA 0.9628 0.841 Optimal H 2 0.5943 0.4621 Balanced truncation (BT) 0.9393 0.6466 Riccati Balanced Truncation (PRBT) 0.9617 0.8164 Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 35 / 38
Outline Summary: Lectures II and III 1 Krylov approximation methods 2 The Arnoldi and the Lanczos procedures The Arnoldi procedure The Lanczos procedure An example 3 Krylov methods and moment matching Remarks 4 Rational interpolation by Krylov projection Realization by projection Interpolation by projection 5 Choice of Krylov projection points: Optimal H 2 model reduction 6 Summary: Lectures II and III Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 36 / 38
Summary: Lectures II and III Approximation methods: Summary Krylov Realization Interpolation Lanczos Arnoldi Properties numerical efficiency n 10 3 choice of matching moments SVD Nonlinear systems Linear systems POD methods Balanced truncation Empirical Gramians Hankel approximation Krylov/SVD Methods Properties Stability Error bound n 10 3 Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 37 / 38
Summary: Lectures II and III Complexity considerations Dense problems Major cost Balanced Truncation: Compute gramians 30N 3 (eigenvalue decomp.) Perform balancing 25N 3 (sing. value decomp.) Rational Krylov approximation: Decompose (A σ i E) for k points 2 3 kn3 Remark : Iterations (Sign, Smith) can accelerate the computation of gramians (esp. on parallel machines) Approximate and/or sparse decompositions Major cost Balanced Truncation: Compute gramians c 1 αkn Perform balancing O(n 3 ) Rational Krylov approximation: Iterative solves for (A σ i E)x = b c 2 kαn, where k = number of expansion points; α = average number of non-zero elements per row in A, E. Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 38 / 38