Zentrum für Technomathematik

Transcription

1 Zentrum für Technomathematik Fachbereich 3 Mathematik und Informatik Necessary optimality conditions for H 2 -norm optimal model reduction Georg Vossen Angelika Bunse-Gerstner Dorota Kubalińska Daniel Wilczek Report Berichte aus der Technomathematik Report Oktober 2007

2

3 NECESSARY OPTIMALITY CONDITIONS FOR H 2 -NORM OPTIMAL MODEL REDUCTION G. VOSSEN, A. BUNSE-GERSTNER, D. KUBALINSKA AND D. WILCZEK Abstract. This paper deals withh 2 -norm optimal model reduction for linear time invariant continuous MIMO systems. We will give an overview on several representations of linear systems in state space as well as in Laplace space and discuss theh 2 -norm for continuous MIMO systems with multiple poles. On this basis, necessary optimality conditions for theh 2 -norm optimal model reduction problem are developed. Key words. Model reduction,h 2 -norm, necessary optimality conditions AMS subject classifications.. Problem formulation. Consider the following linear time invariant (LTI descriptor system ( { } A B ẋ(t Ax(t + Bu(t, Σ : : (. C 0 y(t Cx(t, where x n, u m and y p are called the state variable, the input variable and the output variable, respectively. The matrices A n,n, B n,m and C p,n are constant matrices w.r.t. the time variable t. The system Σ is referred to as stable if all eigenvalues of A are in the left half complex plane (LHP, i.e., all eigenvalues λ j of A satisfy Re(λ j < 0. Let us now introduce the reachability matrix R n and the observability matrix O N defined by R n (Σ [B, AB,..., A n B] n,nm, O n (Σ [C, A C,...,(A n C ] pn,n. A system is called reachable, resp., observable if R n, resp., O n has full rank n. In this paper we assume that all occuring systems are stable, reachable and observable. The goal of model reduction is to find a reduced system ˆΣ : ˆx(t ˆx(t + ˆBu(t, ŷ(t Ĉˆx(t (.2 with ˆx r,  r,r, ˆB r,m and Ĉ p,r with the property that a certain norm of the so-called error system A 0 B Σ ˆΣ 0  ˆB C Ĉ 0 is small. In this paper, we are interested in optimal model reduction with respect to the H 2 -norm of the system which is defined as follows. The Laplace transform L{f(t}(s 0 f(te st dt applied to the system (. leads to a purely algebraic system of equations in the frequency domain: sx(s x(0 AX(s + BU(s, Y (s CX(s (.3

4 2 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek where X(s L{x(t}(s, U(s L{u(t}(s and Y (s L{y(t}(s denote the Laplace transform of the state, input and output, respectively. For simplicity, we assume x(0 0. Solving system (.3 for Y leads to where Y (s H(sU(s H(s C(sI n A B, s (.4 with I n being the n-th order identity matrix is called the transfer function. The H 2 - norm of the system is Σ is defined as the H 2 -norm of its transfer function H given by H 2 H 2 2π trace(h(iw H(iw dw, (.5 cf., e.g., [] where i is the imaginary number with i 2. Hence, the aim of H 2 -norm optimal model reduction is ˆΣ J(ˆΣ : Σ ˆΣ 2 H 2. (.6 or equivalently, ˆΣ J(Ĥ H Ĥ 2 H 2. (.7 We note that representation (. of the system Σ is not unique as the state basis can be transformed by x S x with any regular matrix S n,n which yields x(t Ā x(t + Bu(t, y(t C x(t (.8 with Ā S AS, B S B and C CS where A and Ā have the same eigenvalues. However, the transfer function for (.8 is the same as for (.. Therefore, it is common to identify the system not with its matrices A, B and C but with its transfer function H and hence, also to speak of H as a system. Definition. (Real system. A system H is called real if there exist real matrices A, B and C such that H C(s Id A B holds where Id is the identity mapping. In the following, we will also use the function H defined by H(s : H(s B (si n A C, s. (.9 2. Properties of a transfer function. 2.. SISO. We will now introduce different representations of a transer function H. On the one hand, H can be written as a quotient of two polynomials: n k0 H(s α ks k n k0 β ks, β k n, (2. with complex coefficients α k, k 0,...,n and β k, k 0,...,n. Here, the zeros of the denominator are the poles of the system, resp., the eigenvalues of the matrix

5 Necessary optimality conditions in model reduction 3 A in (.. On the other hand, around each pole λ j, j,...,n, we can expand H into its Laurent series, i.e., H(s γ jl (s λ j l. (2.2 l n Here, the Laurent series starts at l n as H is a rational function with the denominator s polynom degree being equal to n. The coefficients γ jl are called the Laurent coefficients of H at λ j and γ j, is called the residue denoted by Res(H, λ j. The order l 0 (j of a pole λ j is defined as the highest index l such that γ jl 0 holds for all l l 0 (j. The case l 0 (j will be referred to as simple pole, for l 0 (j < we use the notation multiple pole. We note that n l 0 (j holds as H is rational. We will now give a third representation based on partial frations which will be helpful in the following proofs Simple poles. In many applications, the system has n different simple poles, i.e., l 0 (j holds for j,...,n. Therefore, it is worth to investigate this situation in more detail. By expanding the transfer function H into partial fractions, we obtain H(s n φ j s λ j, φ j 0, j,...,n, (2.3 where the pairwise different λ j are the poles of the system and φ j Res(H, λ j are the residues at λ j, j,...,n. Obviously, the matrices A, B and C defined by A diag[λ,...,λ n ], B [... ], C [φ,..., φ n ] (2.4 describe the system (2.3. Hence, the entries c j of the vector C in the representation (2.4 are the residues of H at the poles λ j. Proposition 2.. The following three conditions are equivalent: (i H is real, (ii α k, k 0,...,n, and β k, k 0,...,n, in (2. are real, (iii the poles λ j, j,..., n, and the residues φ j, j,...,n, in (2.3 appear in conjugate pairs, i.e., if λ j is a pole with residue φ j, then λ j is a pole with residue φ j. Proof: Implication (i (ii is obvious. For implication (ii (iii, equation (2. yields H( s n k0 α k s k n n k0 β k s k k0 ᾱks k n β k0 k s H(s k as α k and β k are real. On the other hand, (2.3 implies Hence, we obtain H( s H(s n n φ j s λ j φ j s λ j n n φ j s λ j. φ j s λ j

6 4 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek and comparison of coefficients leads to (iii. For implication (iii (i we will now give a representation with real matrices A, B and C if poles and residues of H appear in conjugate pairs. Without loss of generality, we consider the case n 2. The proof for n > 2 can be done blockwise. If we have two different real poles with due to condition (iii real residues, one can take representation (2.4. For nonreal poles λ λ 2 : λ with residues φ φ 2 : φ the transfer function is given by H(s φ s λ + φ s λ. Consider the system represented by the matrices ( ( Reλ Im λ A, B, C (Re φ Im φ, Re φ + Im φ. (2.5 Imλ Re λ Its transfer function is given by ( C(sI 2 A s Re λ Im λ B (Re φ Im φ, Re φ + Im φ Im λ s Re λ 2 (Re φ(s Re λ (Im φ(im λ (s Re λ 2 + (Im λ 2 φ s λ + φ s λ H(s. ( This completes the proof. We note that the well-known reachable (and observable canonical form is also a proof for impication (ii (i. The funtion H is given by H(s H(s n φ j s λ j (2.6 with φ j and λ j from (2.3. Due to Proposition 2., we obtain H H for real systems Multiple poles. In this case, the transfer function has the following form: H(s n N j l φ N jl (s λ j l, n j n, φ j,nj 0, j,..., N, (2.7 where the pairwise different λ j, j,...,n, are the poles, each of order n j with corresponding coefficients φ jl, l,..., n j. For j N, rewrite (2.7 as H(s N n k k l φ N kl (s λ k l k,k j n k l φ kl (s λ k l + n j l φ jl (s λ j l. The first sum is holomorphic around each λ j. Therefore, it can be expanded into the Taylor series around λ j which coincides with all summands of the Laurent series of H(s around λ j with nonnegative indices. The second term corresponds to all summands of the Laurent series with negative indices, i.e., φ jl γ j, l for l,...,n j. This term is often called principal part of the Laurent series around λ j and we will therefore denote the coefficients φ jl, l,...,n j as principal coefficients. For a closer investigation of the principal coefficients define the Jordan matrices J j, j,...,n,

7 Necessary optimality conditions in model reduction 5 as the n j n j -dimensional matrix with λ j on each diagonal, ones on the super-diagonal and zeros elsewhere. Due to (s λ j... (s λ j nj (si n J j...., (2.8 0 (s λ j the following matrices A, B and C describe the system (2.7: A diag[j,...,j N ], B [e n,..., e n N ], C [φ,n,...,φ,..., φ N,nN,...,φ N,n ]. (2.9 Here, e j is the j-th unity vector, i.e., e j (0,..., 0, j. Therefore, the entries of the matrix C in representation (2.9 are the principal coefficients φ jl in representaion (2.7 of H. Proposition 2.2. The following three conditions are equivalent: (i H is real, (ii α k, k 0,...,n, and β k, k 0,...,n, in (2. are real, (iii the poles λ j, j,...,n, and corresponding principal coefficients φ jl, j,...,n, l,...,n j, in (2.7 appear in conjugate pairs, i.e., if λ j is a pole with coefficients φ jl, then λ j is a pole with coefficients φ jl. Proof: Again, (i (ii is obvious and (ii (iii was already proven in Proposition 2.. For (iii (i, it is without loss of generality sufficient to find real matrices A, B and C which represent the following transfer function H with two nonreal poles λ λ 2 : λ and corresponding principal coefficients φ φ 2 : φ, resp., φ 2 φ 22 : ψ: H(s φ s λ + φ s λ + Consider the system represented by ( A I A 2, B 0 A ψ (s λ 2 + ψ (s λ 2. ( B B 2, C (C, C 2. where ( Re λ Im λ A Im λ Re λ ( 0, B 0 C (Re φ Im φ, Re φ + Im φ, (, B 2, C 2 (Re ψ Im ψ, Re ψ + Im ψ. Its transfer function is C(sI 4 A B C ( (si2 A (si 2 A 2 0 (si 2 A B C (si 2 A 2 B 2 + C 2 (si 2 A B 2 (Re φ(s Reλ (Im φ(im λ 2 (s Re λ 2 + (Im λ (Re ψ( (s Re λ 2 Im λ 2(Im ψ(im λ(s Re λ ((s Re λ 2 + (Im λ 2 2

8 6 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek which completes the proof. The funtion H is given by H(s n N j l φ jl (s λ j l. Equivalently to the SISO case, Proposition 2.2 yields H H for real systems MIMO. Now, B and C are matrices: B [B... B m ], C [(C... (C p ], with column vectors B i (B, i..., Bn i T n representing the i-th input for i,...,m and row vectors C k (C k,..., Ck n n representing the k-th output for k,...,p of the system. Therefore, the transfer function H is a (p m-dimensional matrix with components H ki, i,...,m, k,...,p: H(s H (s... H m (s H p (s... H pm (s, H ki (s C k (si n A B i, (2.0 and each H ki can be seen as a SISO transfer function with input B i and output C k Simple poles. Here, each H ki has the form H ki (s n φ ki j s λ j, (2. where λ j, j,...,n, are the poles of H ki (which are the same for each H ki and φ ki j, j,...,n are the residues of λ j. We note that for p > and m > it is not possible to give representations A, B and C for the system as in (2.4 with B being a constant vector. Hence, the residues now depend on both B and C. Consider a representation where A is diagonalized, i.e., A diag[λ,...,λ n ]. By definition of H, each component H ki is given by H ki (s n C k j Bi j s λ j and comparison of coefficients yields that the residues satisfiy φ ki j Cj k Bi j, i,...,m, k,..., p, j,...,n. ( Multiple poles. Here, the entries H ki of H have the following representations: H ki (s n N j l φ ki N jl (s λ j l, n j n (2.3 with φ ki jl being coefficients in the principal part of Laurent series of H ki around the poles λ j of order n j, j...,n. For a better understanding of the residues we consider a Jordan representation of A, i.e. A diag[j,..., J N ] (cf. the SISO case.

9 Necessary optimality conditions in model reduction 7 We will first investigate the case of one pole λ of order n. Then, we have A J and (2.8 leads to a transfer function with components H ki (s n Cl k l n ql Bq i n (s λ q l+ l n l+ q C k q Bi q+l (s λ l. (2.4 For N > poles, we divide B ( B,..., B N with matrices B j n j,m and C ( C,..., C N with matrices C j p,n j for j,...,n. Due to H(s N C j (si n J j Bj, equation (2.4 implies that each component of H is given by H ki (s n N j l n l+ q ( C j k q( B j i q+l (s λ j l. Hence, we obtain the following representation for the residues: n l+ φ ki jl ( C j k q( B j i q+l. (2.5 q 3. The H 2 -norm. We will now give explicite formulas how to derive the H 2 - norm of a given system H. Due to (.9, the H 2 -norm becomes to H 2 H 2 2π ( trace H( iwh(iw dw 2πi i i 3.. SISO. As in this case H is scalar, (3. simplifies to H 2 H 2 2πi i i ( trace H( zh(z dz. (3. H( zh(zdz. (3.2 We will calculate this complex integral via the classical Residue Theorem from complex analysis (cf., e.g. [2] which we state here in a simple version suitable for our purpose. Theorem 3. (Residue Theorem. Let F : be a complex meromorphic function and γ be Jordan curve, i.e., a simple closed curve, in such that F has no poles on γ. Then, the following holds: 2πi γ F(zdz Res(F, λ. (3.3 λ pole of F inside γ Here and in the following, a point is referred to lie inside a curve if its winding number is equal to one. For example, each point z with z < is inside the anticlockwise parametrized unit circle, i.e., γ(t : e 2πit, t [0, ]. We note that the sum in (3.3 is finite as the function F is meromorphic. The following result will be helpful for the computation of formula (3.2. Proposition 3.2. Let F : be a rational function of order o 2, i.e., N k0 F(z α kz k M k0 β kz, α N, β M 0, o : N M 2, k

10 8 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek ir -R γ 2 γ -ir Fig. 3.. The curves γ (dashed line and γ 2 (solid line with no poles on the imaginary axis. Then the following holds: 2πi i i F(zdz Res(F, λ. λ pole of F in LHP Proof: Consider for a fixed R > 0 the Jordan curve γ defined by { γ (t, t [0, ], γ (t : ir + 2iRt, t [0, ], γ(t : γ 2 (t, t [, 2], γ 2 (t : ire πit, t [0, ], for t [0, 2] as shown in Figure 3.. Then, we have ir ir F(zdz F(zdz γ γ F(zdz F(zdz. γ 2 (3.4 The first integral can be computed via the Residue theorem: F(zdz Res(F, λ Res(F, λ. 2πi γ R λ pole of F inside γ λ pole of F in LHP The second integral in (3.4 is bounded by F(zdz γ 2 max F(z πr max F(z. γ z γ 2 2 z R For sufficiently large R, there exists a constant c such that F(z c z N M holds for z R which implies F(zdz cπr N M+ cπ γ 2 R 0. R This completes the proof. Due to (2., H( zh(z is a complex rational function of order o 2. Since for a stable system H, the function H( z has no poles in LHP, the function H( zh(z has the same poles in LHP as H(z and the H 2 -norm becomes to ( H 2 H 2 Res H( zh(z, z λ. (3.5 λ pole of H(z in LHP

11 Necessary optimality conditions in model reduction Simple poles. The following result is well-known in complex analysis and can be found, e.g., in [2]. Proposition 3.3. Let F be a meromorphic complex function and λ be a simple pole of F with residue φ. Let furthermore G be a complex function which is holomorphic in λ. Then, we have Res(F G, λ Res(F, λg(λ φg(λ. This brings us to the main result in this paragraph. Proposition 3.4. The H 2 -norm of a stable system H with simple poles λ j and corresponding residues φ j, j,..., n, is given by If H is real, we have H 2 H 2 n φ j H( λ j H 2 H 2 n φ jh( λ j. (3.6 n φ j H( λ j. (3.7 Proof: Using equation (3.5, we directly obtain (3.6 where the second equality arises from the fact that holds. Furthermore, Proposition 2. yields ( Multiple poles. Due to (3.5, we have to calculate residues of H( zh(z in λ j, j,...,n, which are now poles of order n j, respectively. We can use the following well-known result, cf., [2]. Proposition 3.5. Let F be a meromorphic complex function and λ be a pole of order k of F. Then, the residue of the function F in λ is given by Res(F, λ ( ((z (k! lim λ k F(z (k. (3.8 z λ where the superscript (k denotes the (k -th derivative with respect to z. In the following, we will calculate the right hand side of equation (3.8 for F H. Lemma 3.6. For j,...,n and l 0,...,n j we have ( lim ((z λ j nj H(z (nj l φ j,l+ (n j l! (3.9 z λ j Proof: The term ( (z λ j nj H(z (n j l can be written as N n i i,i j k φ ik (z λ j nj (z λ i k (n j l + ( nj (nj l φ jk (z λ j nj k. k Here, the first summand vanishes for z λ j since ( ((z λj nj (z λ i k (n j l zλj 0 holds, whereas for the second summand we obtain ( l+ φ jk (n j l!(z λ j zλj +l k φ j,l+ (n j l! k

12 0 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek Now we come to the main result of this paragraph. Proposition 3.7. The H 2 -norm of a stable system H with simple poles λ j and corresponding principal coefficients φ jl, j,...,n, l,...,n j, is given by H 2 H 2 n N j l If H is real, we have ( l (l! φ jlh (l ( λ j H 2 H 2 n N j l n N j l Proof: Using Proposition 3.5 and Lemma 3.6, we obtain Res( H( zh(z, λ j ( ( H( zh(z (nj lim (z λ j nj (n j! z λ j (n j! (n j! (n j! n j l lim z λ j n j l0 n j l0 ( nj l0 ( nj l ( nj l ( nj l ( l (l! φ jl H (l ( λ j. ( l (l! φ jlh (l ( λ j. (3.0 ( l (l! φ jlh (l ( λ j. (3. ((z λj nj H(z (n j l( H( z (l ( ((z lim λj nj H(z (n j l ( (l H( z zλj z λ j φ j,l+ (n j l! ( l H(l ( λ j Equation (3.5 directly yields (3.0, and (3. can be obtained from Proposition MIMO. The H 2 norm is given by H 2 H 2 trace(h(iw H(iw dw (H ki (iw H ki (iw dw 2π 2π i,k H ki (iw H ki (iwdw H ki 2 H 2π 2 i,k i,k (3.2 and hence, the results from the previous section about SISO systems can be used to compute the H 2 -norm of a MIMO system Simple poles. Due to (3.6, for each H ki the norm is given by H ki 2 H 2 n (φ ki j H ki ( λ j

13 which, together with (3.2, implies Necessary optimality conditions in model reduction H 2 H 2 i,k n n j H ki ( λ j (φ ki i,k (φ ki j H ki ( λ j ( Multiple poles. Due to (3.0 and (3.2, the norm is H 2 H 2 n N j l i,k 4. Necessary optimality conditions. 4.. SISO. ( l (l! (φki jl H (l ki ( λ j ( Simple poles. As mentioned in the previous section, for simple poles it is possible to find a representation A, B and C of a system such that B e n holds and the only free paramters are the poles of Σ on the diagonal of A and the residues in C. Hence, minimization problem (.7 can be seen as v J(v H Ĥ 2 H 2. (4. with the optimization variable v (ˆλ,..., ˆλ r, ˆφ,..., ˆφ r T 2r. (4.2 If we assume that the poles of the original system are different from the poles of the reduced system, the error system H Ĥ has n + r simple poles λ,...,λ n, ˆλ,..., ˆλ r with corresponding residues φ,...,φ n, ˆφ,..., ˆφ r. Formula (3.6 implies that the cost functional in (4. can be written as J(v n φ j (H( λ j Ĥ( λ j + r ˆφ j (Ĥ( ˆλ j H( ˆλ j which yields a nonsmooth optimization problem with the complex optimization vector v 2r. We consider three different types of optimization problems: Case : v J(v subject to v 2r, (4.3 i.e., the optimization vector v shall real. This is a reasonable task if the original system H is real or even has real poles and residues. In this case, due to (3.7, all superscripts can be deleted in J and we obtain J(v n φ j (H( λ j Ĥ( λ j + r ˆφ j (Ĥ( ˆλj H( ˆλ j. (4.4 Furthermore, instead of adding the constraint v 2r, one can also directly see the problem as a real unconstrained one. Therefore, we obtain a smooth unconstraint optimization problem with the real optimization vector v 2r. Theorem 4.. Necessary optimality conditions for problem (4.3 are given by H( ˆλ j Ĥ( ˆλ j, H ( ˆλ j Ĥ ( ˆλ j, j,...,r, (4.5

14 2 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek i.e., interpolation of Ĥ and its first derivative with H at ˆλ j, j,...,r. This result together with a proof was already given by Meier/Luenberger [4] and Gugercin/Antoulas/Beattie [3]. However, we note that in both references the authors did not mention that their proof is only applicable in case, i.e., for real poles and residues, as for complex v one has to the cost functional J in (4. which is not differentiable w.r.t. the optimization variable. We furthermore note that, nevertheless, in both publications the result is claimed to be correct and used in examples also for nonreal poles and residues. We will now investigate in which cases the result is still true for a complex v and when it should slightly be modified. Case 2: v J(v, (4.6 i.e., there shall be no constraints. The typical approach for such nonsmooth optimization problems with a complex variable v 2r 4r is to consider the real variable ṽ consisting of real and imaginary parts of the components of v as optimization vector in the general case: ṽ (Re ˆλ, Im ˆλ,..., Re ˆλ r, Im ˆλ r, Re ˆφ, Im ˆφ,..., Re ˆφ r, Im ˆφ r T 4r (4.7 Then, J is smooth w.r.t. ṽ and hence, we obtain a smooth unconstrained optimization problem with optimization variable ṽ 4r. For the statement of the main result in case 2, the following lemma will be helpful. Lemma 4.2. (i For µ,...,r we have: H( λ j Re ˆφ µ H( λ j Im ˆφ µ H( ˆλ j Re ˆφ µ H( ˆλ j Im ˆφ µ 0, Ĥ( λ j Re ˆφ µ i Ĥ( λ j Im ˆφ µ (ii For µ,...,r we have: H( λ j Re ˆλ µ H( λ j Im ˆλ µ 0, H( ˆλ j Re ˆλ µ i Ĥ( ˆλ j Re ˆλ µ Ĥ( ˆλ j Im ˆλ µ λ j ˆλ µ, H( ˆλ j Im ˆλ µ ˆφ µ (ˆλ j +ˆλ µ Ĥ ( ˆλ j + i ˆφ µ (ˆλ j +ˆλ µ iĥ ( ˆλ j Ĥ( ˆλ j Re ˆφ µ i Ĥ( λ j Re ˆλ µ i { 0, j µ, H ( ˆλ j, j µ, 2, j µ, ˆφµ (ˆλ j +ˆλ µ 2, j µ, 2, j µ, ˆφµ (ˆλ j +ˆλ µ 2, j µ. Ĥ( ˆλ j Im ˆφ µ Ĥ( λ j Im ˆλ µ ˆλ j ˆλ µ. ˆφ µ (λ j + ˆλ µ 2, Theorem 4.3. Necessary optimality conditions for problem (4.6 are given by H( ˆλ j Ĥ( ˆλ j, H ( ˆλ j Ĥ ( ˆλ j, j,..., r, (4.8 i.e., interpolation of Ĥ and its first derivative with H at ˆλ j, j,...,r.

15 Necessary optimality conditions in model reduction 3 Note that ˆλ j, j,...,r, are called mirror images. Proof: In view of Lemma 4.2, differentiating J with respect to Re ˆφ µ yields Re ˆφ µ n φ j λ j ˆλ + Ĥ( ˆλ µ + µ r H( ˆλ µ + Ĥ( ˆλ µ + Ĥ( ˆλ µ H( ˆλ µ H( ˆλ µ + Ĥ( ˆλ µ + Ĥ( ˆλ µ H( ˆλ µ 2Re(Ĥ( ˆλ µ H( ˆλ µ. ˆφ j ˆλ j ˆλ µ H( ˆλ µ (4.9 Analogously, we obtain Im ˆφ ih( ˆλ µ iĥ( ˆλ µ + iĥ( ˆλ µ + ih( ˆλ µ µ 2Im (Ĥ( ˆλ µ H( ˆλ µ. (4.0 In view of equations (4.9 and (4.0, the necessary optimality condition / z 0 implies the first condition in (4.8. Differentiation of J with respect to real, resp., imaginary part of ˆλ µ yields n ˆφ Re ˆλ φ µ r j µ (λ j + ˆλ µ + ˆφ ˆφ µ 2 j,j µ (ˆλ j + ˆλ µ 2 ( + Ĥ ( ˆλ µ + ˆφµ (ˆλ µ + ˆλ ˆφ µ 2 µ + ˆφ µ H ( ˆλ µ ˆφ H µ ( ˆλ µ ˆφ Ĥ µ ( ˆλ µ ˆφ µĥ ( ˆλ µ + ˆφ µh ( ˆλ µ 2Re (ˆφ µ (H ( ˆλ µ Ĥ ( ˆλ µ, n ˆφ Im ˆλ i φ µ r j µ (λ j + ˆλ µ + i ˆφ ˆφ µ 2 j,j µ (ˆλ j + ˆλ µ 2 ( ˆφ + i Ĥ ( ˆλ µ µ + (ˆλ µ + ˆλ ˆφ µ 2 µ ˆφ µh ( ˆλ µ 2Im (ˆφ µ (H ( ˆλ µ Ĥ ( ˆλ µ. (4. In view (4., / z 0 implies the second condition in (4.8 which completes the proof. Remark 4.4. If the resulting system in problem (4.6 for case 2 turns out to be real, corresponding optimality conditions (4.8 are equivalent to conditions (4.5 for case. Case 3: v J(v subject to Ĥ is real. (4.2 This case is reasonable if already the original system H is real what we will assume in the following. Now, v can be complex but all poles and residues appear in conjugate pairs as shown in Proposition 2.. For simplicity of notation, we assume that all

16 4 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek components of v are purely imaginary and hence, that r 2R, R, is even. For all real components one may use the results of case. Then, (4.2 is equivalent to v J(ṽ subject to ˆλj ˆλ R+j, ˆφj ˆφ R+j, j,...,r. (4.3 This can be seen as a constrained smooth optimization problem using again ṽ from (4.7 as optimization vector. Theorem 4.5. Necessary optimality conditions for problem (4.2 are given by H( ˆλ j Ĥ( ˆλ j, H ( ˆλ j Ĥ ( ˆλ j, j,...,r. Proof: The Lagrangian function in normal form for (4.3 is given by L(ψ, χ, v J(v + R Re ψ j (Re ˆλ j Re ˆλ R R+j + Im ψ j (Im ˆλ j + Im ˆλ R+j R + Re χ j (Re ˆφ j Re ˆφ R R+j + Im χ j (Im ˆφ j + Im ˆφ R+j with the complex Lagrange multipliers ψ (ψ,...,ψ R, χ (χ,...,χ R view of (4.9, for µ,...,r we obtain L(ψ, χ, v Re ˆφ µ R. In 2Re (Ĥ( ˆλ µ H( ˆλ µ + Re χ µ (4.4 and L(ψ, χ, v Re ˆφ 2Re (Ĥ( ˆλ R+µ H( ˆλ R+µ Re χ µ R+µ (4.5 2Re (Ĥ( ˆλ µ H( ˆλ µ Re χ µ since ˆλ j ˆλ R+j holds and H and Ĥ are real. Necessary optimality conditions for (4.3 imply that L(ψ, χ, v/ ṽ vanishes. Setting equations (4.4 and (4.5 to zero and subtracting them, we obtain Re χ µ 0 for j,..., R. Analogously, one can see that Im χ µ Re ψ µ Im ψ µ 0 holds which means that all Lagrange multipliers vanish and we obtain the same necessary conditions as in (4.8, resp., (4.5 since H and Ĥ are real Multiple poles. Due to (2.9, for any system H it is always possible to find a representation A, B and C with the same matrix B and only the poles and the principal coefficients as free variables. Hence, the optimization variable for problem (.7 in the case of multiple poles is given by v (ˆλ,..., ˆλ R, ˆφ,..., ˆφ,r,..., ˆφ R,..., ˆφ R,rR T R+r (4.6 where R is the given and fixed number of poles in the reduced system, each of given and fixed order r j, with corresponding principal coefficients, φ jl, j,...,r, l,...,r R, r r R r. Assuming again that all poles of the original system are different from the poles of the reduced system, the error system H Ĥ has n + R simple poles λ,..., λ n, ˆλ,..., ˆλ R with corresponding coefficients

17 Necessary optimality conditions in model reduction 5 φ,...,φ N,nN, ˆφ,..., ˆφ R,rR. Due to (3.0, we have the following cost functional: J(v n N j l + r R j l ( l (l! φ jl (H (l ( λ j Ĥ(l ( λ j ( l (l! ˆφ (Ĥ(l jl ( λ j H (l ( λj. As for simple poles, we will consider the following three cases. Case : Case 2: Case 3: v v v J(v subject to v 2r, (4.7 J(v, (4.8 J(v subject to Ĥ is real. (4.9 We begin with the general case 2 and consider again the real optimization variable ṽ 2(R+r consisting of real and imaginary parts of the components of v. The following result can be proved analogously to Lemma 4.2 in the case of simple poles. Lemma 4.6. (i For µ,...,r, ν,...,r µ we have: H (l ( λ j Re ˆφ µν H(l ( λ j Im ˆφ µν H(l ( ˆλ j Re ˆφ µν H(l ( ˆλ j Im ˆφ µν 0, Ĥ(l ( λ j Re ˆφ µν i Ĥ(l ( ˆλ j Re ˆφ µν i (ii For µ,...,r we have: Ĥ(l ( λ j Im ˆφ µν ( ν (ν + k 2! (ν!(λ j + ˆλ µ ν+l, Ĥ(l ( ˆλ j Im ˆφ µν ( ν (ν + k 2! (ν!(ˆλ j + ˆλ µ ν+l. H (l ( λ j Re ˆλ µ H(l ( λ j Im ˆλ µ 0, Ĥ(l ( λ j Re ˆλ Ĥ(l ( λ j ( k (k + l!ˆφ µk µ i Im ˆλ µ k (k!(λ j + ˆλ, µ k+l H (l ( ˆλ j Re ˆλ H (l { ( ˆλ j 0, j µ, µ i Im ˆλ µ H (k ( ˆλ j, j µ, r µ Ĥ(l ( ˆλ j Re ˆλ µ Ĥ(l ( ˆλ j Im ˆλ µ k r µ ( k (k+l!ˆφ µk (k!(ˆλ j +ˆλ µ k+l, j µ, Ĥ(l ( ˆλ j + r µ r µ k k ( k (k+l!ˆφ µk (k!(ˆλ j +ˆλ µ k+l, j µ. ( k (k+l!ˆφ µk (k!(ˆλ j +ˆλ µ k+l, j µ, iĥ(l ( ˆλ j r µ k ( k (k+l!ˆφ µk (k!(ˆλ j +ˆλ µ k+l, j µ.

18 6 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek Theorem 4.7. Necessary optimality conditions for problem (4.8 are given by H (l ( ˆλ j Ĥ(l ( ˆλ j, j,...,r, l 0,...,r j (4.20 i.e., interpolation of Ĥ and its first r j derivative with H at ˆλ j, j,...,r. Proof: Similar to the case of simple poles and using Lemma 4.6, differentiating J with respect to ṽ yields Re ˆφ 2 ( ν µν (ν! Re ( ˆλ µ H(ν ( ˆλ µ, (4.2 Im ˆφ 2 ( ν µν (ν! Im ( ˆλ µ H(ν ( ˆλ µ (4.22 for µ,...,r and ν,...,r µ and Re ˆλ µ 2 Im ˆλ µ 2 r µ l r µ l ( l ( (ˆφ (l! Re µl H (l ( ˆλ µ Ĥ(l ( ˆλ µ, ( l ( (ˆφ (l! Im µl H (l ( ˆλ µ Ĥ(l ( ˆλ µ (4.23 for µ,...,r. Equations (4.2 and (4.2 gives interpolation of the first r j derivatives which, together with (4.23, implies interpolation of r j -th derivatives in each ˆλ j, j,...,r. For case, one can use the previous proofs. We obtain all derivative formulas w.r.t. v by omitting the Re notation in all preceding formulas for derivatives w.r.t. the real parts of the components of ṽ. This directly yields the optimality conditions (4.20. Using Proposition 2.2, we obtain the following result. Theorem 4.8. Necessary optimality conditions for problem (4.7 are given by H (l ( ˆλ j Ĥ(l ( ˆλ j, j,...,r, l 0,...,r j i.e., interpolation of Ĥ and its first r j derivative with H at ˆλ j, j,...,r. Case 3 can be handled as for simple poles. Again, we obtain that the Lagrangian multipliers of problem (4.9 vanish and we obtain the following result. Theorem 4.9. Necessary optimality conditions for problem (4.9 are given by H (l ( ˆλ j Ĥ(l ( ˆλ j, j,...,r, l 0,...,r j i.e., interpolation of Ĥ and its first r j derivative with H at ˆλ j, j,...,r MIMO. For MIMO systems, we will only consider the general case (i.e., case 2 for SISO of optimization problem (.7 without constraints. Contrary to the SISO case, the residues of a transfer function now depend on both B and C if A is given in eigenvalue decomposition form. Hence, both ˆB and Ĉ act as optimization variables additionally to the eigenvalues of  Simple poles. The optimization vector is now given by v (ˆλ,..., ˆλ r, ˆB,..., ˆB m r, Ĉ,...,Ĉp r T r+mr+pr, (4.24

19 Necessary optimality conditions in model reduction 7 resp., ṽ (Re v T, Imv T T 2(r+mr+pr. Similarly to the SISO case we consider the error system H Ĥ which has the n + r simple poles λ,...,λ n, ˆλ,..., ˆλ r with corresponding residues φ ki,..., φki ki ki n, ˆφ,..., ˆφ r in the ki-th component of the error system. In view of (3.3 and (2.2, the optimization problem can be formulated as v J(ṽ n i,k + ( j H ki ( λ j Ĥki( λ j (φ ki r ( (ˆφ ki j Ĥ ki ( ˆλ j H ki( ˆλ j i,k (4.25 which is smooth with respect to ṽ and unconstrained. Theorem 4.0. Necessary optimality conditions for problem (4.25 are given by where H( ˆλ j ˆB j Ĥ( ˆλ j ˆB j, Ĉ j H( ˆλ j Ĉ j Ĥ( ˆλ j, Ĉ j H ( ˆλ j ˆB j Ĉ j Ĥ ( ˆλ j ˆB j, j,...,r, (4.26 Ĉ j : j-th column of Ĉ, ˆBj : j-th row of ˆB. In other words, the optimal reduced transfer function Ĥ is a left sided tangential interpoland into the directions Cj, a right sided tangential interpoland into the directions Bj and a both sided tangential interpoland into the directions C j, resp., B j of the original transfer function H. Proof: Since J can be written as J i,k J ki with n ( r ( J ki (φ ki j H ki ( λ j Ĥki( λ j + (ˆφ ki j Ĥ ki ( ˆλ j H ki( ˆλ j we can use the derivative formulas from the SISO case for each J ik to calculate the derivatives of J. Define the vectors ˆφ ki : (ˆφ ki ki,..., ˆφ r T r, ˆφ : (ˆφ,..., ˆφ pm rpm. Then, we have Re ˆφ ( Re ˆφ,..., pm, Re ˆφ pm ( Im ˆφ Im ˆφ,..., pm Im ˆφ pm with entries as in the SISO case given in (4.9 and (4.0. Due to (2.2, the residues satisfy Ĉk ˆB j j i, resp., Re ˆφ ki j ki ˆφ j Re Ĉk j Re ˆB j i Im Ĉk j Im ˆB j, i ki Im ˆφ j Re Ĉk j Im ˆB j i + Im Ĉk j Re ˆB j. i Hence, for i,...,m we obtain Re ˆφ ki j Re Ĉκ µ Re ˆB i µ, Im ˆφ ki j ReĈκ µ Im ˆB i µ, Re ˆφ ki j Im Ĉκ µ Im ˆB i µ, Im ˆφ ki j Im Ĉκ µ Re ˆB i µ

20 8 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek if κ k and µ j. For κ k or µ j, the derivatives are zero. Then, the chain rule yields Analogously, we obtain Reφ ReĈκ µ Re ˆφ ImĈκ µ Re ˆB ι µ Im ˆB ι µ i,k m 2 Re i i Re Ĉκ µ + Im φ Im ˆφ ReĈκ µ ki Re φ ki Re ˆφ + ki Re Ĉκ µ i,k m Im i ki Im φ ki Im ˆφ ki Re Ĉκ µ (Ĥκi ( ˆλ µ H κi( ˆλ µ Re ˆB i µ (Ĥκi ( ˆλ µ H κi ( ˆλ µ Im ˆB i µ m ( Re [(Ĥκi ( ˆλ µ H κi( ˆλ µ ˆB µ i ]. m i p k p k ( Im [(Ĥκi ( ˆλ µ H κi( ˆλ µ ] ˆBi µ, (Ĉk Re [(Ĥkι ( ˆλ µ H kι ( ˆλ ] µ µ, (Ĉk Im [(Ĥkι ( ˆλ µ H kι( ˆλ µ ] µ. Hence, the necessary optimality condition / z 0 imply m Ĥ κi ( ˆλ µ ( m ˆBi µ H κi ( ˆλ µ (, ˆBi µ i i p (Ĉk Ĥ p µ kι ( ˆλ µ (Ĉk Hkι µ ( ˆλ µ, k k κ,...,p, ι,...,m, for µ,...,r which is equivalent to the first two conditions in (4.26. Differentiation of J with respect to real, resp., imaginary part of ˆλ µ via formulas (4. for J ik directly yields ( (ˆφik ( Re µ H ( ˆλ µ Ĥ ( ˆλ µ, Re ˆλ µ 2 i,k Im ˆλ µ 2 i,k ( (ˆφik ( Im µ H ( ˆλ µ Ĥ ( ˆλ µ for µ,...,r. Due to necessary conditions and φ ki j (Ĉk Ĥ µ ki( ˆλ µ ( ˆBi µ Cj kbi j, we obtain (Ĉk µ H ki ( ˆλ µ ( ˆBi µ, µ,..., r. i,k i,k This is equivalent to the third condition in (4.26 and hence, completes the proof.

21 Necessary optimality conditions in model reduction Multiple poles. Now, the optimization problem is given by v J(ṽ n N j l i,k + r R j l i,k ( l (l! (φki ( l ki (ˆφ (l! ( jl H (l ki ( λ j Ĥ(l ki ( λ j ( jl Ĥ (l ki ( ˆλ j H (l ki ( ˆλ j (4.27 with the optimization vector v (ˆλ,..., ˆλ R, ( ˆ B,..., ( ˆ BN m n N, ( ˆ C,...,( ˆ CN p n N T R+mr+pr, (4.28 resp., ṽ (Re v T, Im v T T 2(R+mr+pr and the principal coefficients φ ki jl and as in (2.5. Theorem 4.. Necessary optimality conditions for problem (4.27 are given by r j l q0 l q0 ( q H (q ( ˆλ q! j ( r ( ˆ Bj j l ( q l+q Ĥ (q ( ˆλ q! j ( ( ˆ Bj, l+q q0 (4.29 ( q ( H l ( ˆ Cj (q l q ( ˆλ j q! ( q ( Ĥ ( ˆ Cj (q l q ( ˆλ j q!, for j,...,r, l,...,r j and r j l for j,...,r where r j l ( l (l! ( l (l! q0 l ( H ( ˆ Cj (l q ( ˆλ j ( ( ˆ Bj q+l q0 ˆφ ki jl (4.30 l ( Ĥ ( ˆ Cj (l q ( ˆλ j ( ( ˆ Bj q+l q0 ( ˆ Cj q : q-th column of the matrix ˆ Cj, ( ˆ Bj q : q-th row of the matrix ˆ Bj. Proof: Again, we write J i,k J ik and use the formulas from the SISO case. Due to the principal coefficients representation (2.5, for i,...,m we have Reφ ki jl Re( ˆ Cµ κ ν Re( ˆ Bµ i ν+l if κ k, µ j, ν r j l +, resp., for k,...,p we obtain Reφ ki jl Re( ˆ Bµ ι ν Re ( ˆ Cµ k ν l+

22 20 G. Vossen, A. Bunse-Gerstner, D. Kubalinska and D. Wilczek if ι i, µ j, ν l + r j l +, whereas derivatives for other indices vanish. Similar formulas hold for derivatives of imaginary parts of ˆφ and also for derivatives with respect to imaginary parts of ˆB and Ĉ. Therefore, the chain rule yields Re ( ˆ C µ κ ν Im( ˆ Cµ κ ν Re ( ˆ Bµ ι ν Im( ˆ Bµ ι ν m i m i p r µ ν+ l r µ ν+ l ν k l p ν k l ( l (l! ( l (l! ( l (l! ( l (l! Re [(Ĥ(l κi Im [(Ĥ(l κi Re [(Ĥ(l kι Im [(Ĥ(l kι (( ( ˆλ µ H(l ( ˆλ µ ˆ Bµ i ν+l κi (( ( ˆλ µ H(l ( ˆλ µ ˆ Bµ i ν+l κi (( ( ˆλ µ H(l ( ˆλ µ ˆ Cµ k ] ν l+, kι (( ( ˆλ µ H(l ( ˆλ µ ˆ Cµ k ν l+ kι ]. ], ], As all derivatives have to vanish, we obtain the two conditions in (4.29. Furthermore, from (4.23 we have Re ˆλ 2 µ i,k Im ˆλ 2 µ i,k r µ l r µ l ( l ( ( (l! Re (ˆφ ki µl H (l ki ( ˆλ µ Ĥ(l ki ( ˆλ µ, ( l ( ( (l! Im (ˆφ ki µl H (l ki ( ˆλ µ Ĥ(l ki ( ˆλ µ which, together with the representation (2.5 for the principal coefficients, yields condition (4.30. Remark 4.2. Note that in case of multiple poles we do not have tangential interpolation as necessary conditions but the sum of certain directions multiplied with certain derivatives of H and Ĥ must coincide. This furthermore implies that contrary to the SISO case, we cannot use the first two conditions (4.29 to obtain a simple third condition which involves only r j -th derivatives of the transfer functions. REFERENCES [] A.C. Antoulas, Approximation of large-scale dynamical systems, Advances in Design and Control 6, SIAM, Philadelphia, [2] J.B. Conway, Functions of one complex variable, Springer, 978. [3] S. Gugercin, C. Beattie and A.C. Antoulas, Rational Krylov Methods for OptimalH 2 Model Reduction, ICAM Technical Report, Virginia Tech, 2006 and submitted to SIAM Journal on Matrix Analysis and Applications. [4] L. Meier and D.G. Luenberger, Approximation of Linear Constant Systems, IEEE Trans. Automatic Control. 2 (