Optimal Approximation of Dynamical Systems with Rational Krylov Methods. Department of Mathematics, Virginia Tech, USA

Transcription

1 Optimal Approximation of Dynamical Systems with Rational Krylov Methods Serkan Güǧercin Department of Mathematics, Virginia Tech, USA Chris Beattie and Virginia Tech. jointly with Thanos Antoulas Rice University Workshop on Large-Scale Robust Optimization, August 31-September 2, 2005, Santa Fe, NM 1

2 Outline 1. Introduction and Problem Statement 2. Motivating Examples 3. Rational Krylov-Interpolation Framework 4. An Iterative Rational Krylov Algorithm 5. Inexact Solves in Krylov-based Model Reduction 3

3 Introduction Consider an n th order single-input/single-output system G(s): ẋ(t) = Ax(t) + bu(t) G(s) : G(s) = c(si n A) 1 b y(t) = cx(t) = n(s) d(s) u(t) R: input, x(t) R n : state, y(t) R: output A R n n, b,c T R n. Will assume R(λ i (A)) < 0 Need for improved accuracy = Include more details in the modeling stage In many applications, n is quite large, n O(10 6, 10 7 ), Untenable demands on computational resources = 4

4 Model Reduction Problem: Find ẋ r (t) = A r x r (t) + b r u(t) y r (t) = c r x r (t) G r (s) = c r (si r A r ) 1 b r where A r R r r, b r,c T r R r, with r n such that 1. y y r is small. 2. The procedure is computationally efficient. A B A r B r C D C r D r G r (s): used for simulation or designing a reduced-order controller 5

5 Model reduction of Serkan from n=3 down to r=2 Cascades, Blacksburg, VA

6 Model reduction through projection: a unifying framework. Construct Π = VZ T, where V,Z R n r with Z T V = I r : ẋ r = Z T AV }{{} :=A r x r (t) + Z T b }{{} u(t), y r (t) = }{{} cv x r (t) :=b r :=c r What is the approximation error e(t) := y(t) y r (t)? G(s): Associate a convolution operator S: S : u(t) y(t) = (Su)(t) = (g u)(t) = t g(t τ)u(τ)dτ. g(t) = ce At b for t 0: Impulse response. Transfer function: G(s) = (Lg)(s) = c(si A) 1 b. 6

7 The H Norm : 2-2 induced norm of S: G(s) H = sup u 0 y 2 u 2 = sup u 0 Su 2 = sup G(jw) u 2 2 w R G G r = Worst output error y(t) y r (t) 2 u(t) 2 = 1. The H 2 Norm : L 2 norm of g(t) in time domain: G(s) 2 H 2 = 0 trace[g T (t)g(t)]dt = 1 2π + trace[g (jw)g(jw)]dw G(s) G(jw) (db) G(s) freq (rad/sec) 7

8 Motivating Example: Simulation A Tunable Optical Filter: (Data: D. Hohlfeld, T. Bechtold, and H. Zappe) An optical filter, tunable by thermal means. Silicon-based fabrication. The thin-film filter: membrane to improve thermal isolation Wavelength tuning by thermal modulation of resonator optical thickness The device features low power consumption, high tuning speed and excellent optical performance. 8

9 Modeling: A simplified thermal model to analyze/simulate important thermal issues: 2D model and 3D model Meshed and discretized in ANSYS 6.1 by the finite element methods The Dirichlet boundary conditions at the bottom of the chip. A constant load vector corresponding to the constant input power of of 1 mw for 2D model and 10 mw for 3D model The output nodes located in the membrane Eẋ(t) = Ax(t) + bu(t), y(t) = cx(t) 2D: n = 1668, nnz(a) = 6209, nnz(e) = D: n = , nnz(a) = , nnz(e) =

10 Motivating Example: Control Optimal Cooling of Steel Profiles in a Rolling Mill : Data: Peter Benner 10

11 Different steps in the production process require different temperatures of the raw material. To achieve high throughput, reduce the temperature as fast as possible to the required level before entering the next production phase. Cooling process by spraying cooling fluids on the surface Must be controlled so that material properties, such as durability or porosity, stay within given quality standards Modeled as boundary control of a two dimensional heat equation. A finite element discretization results in Eẋ(t) = Ax(t) + bu(t), y(t) = cx(t). n = 79, 841: nnz(a) : , nnz(e) :

12 Model Reduction via Interpolation Rational Interpolation: Given G(s), find G r (s) so that G r (s) interpolates G(s) and certain number of its derivatives at selected frequencies σ k in the complex plane ( 1) j j! d j G(s) ds j = ( 1)j s=σk j! d j G r (s) ds j, s=σk for k = 1,...,K, and j = 1,...,J ( 1)j j! d j G(s) ds j = c(σ k I A) (j+1) b: s=σk = j th moment of G(s) at σ k. Why to choose model reduction via rational interpolation? 12

13 Generically, any reduced model G r (s) can be obtained via interpolation. Interpolation points = Zeroes of G(s) G r (s). BUT: Prob-1: What is a good selection of interpolation points? Similar to polynomial approximation of complex functions. Recall: Trying to match the moments: c(σ k I A) (j+1) b Moments are extremely ill-conditioned Prob-2: Construct G r (s) without explicit moment computation Prob-2 easier to tackle using rational Krylov framework (Skelton et al. [1987], Grimme [1997]): 13

14 Given r interpolation points: {σ i } r i=1 Set V = Span [ (σ 1 I A) 1 b,, (σ r I A) 1 b ], and Z = Span [ (σ 1 I A T ) 1 c T,, (σ r I A T ) 1 c ] T, Z T V = I r. A r = Z T AV, b r = Z T b, c r = cv = G(σ i ) = G r (σ i ), and d ds G(s) = d s=σi ds G r(s) s=σi Moment matching without explicit moment computation Still to answer: How to choose σ i? σ i = λ i (A) (Antoulas/G [2003]). Effective but not optimal. Does there exist an optimal selection? 14

15 Optimal H 2 approximation Problem: Given a stable dynamical system G(s), find a reduced model G r (s) that satisfies G r (s) = arg G(s) Ĝ(s) H2. min deg(ĝ) = r Ĝ : stable Existence of a global minimal: Exists in the SISO case Not known for the MIMO case General approach: Find G r (s) that satisfies first-order necessary conditions: Wilson [1970], Meier and Luenburger [1967], Hyland and Bernstein [1985], Yan and Lam [1999],... 15

16 Framework of Wilson [1970] Given G r (s) = c r (si r A r ) 1 b r, define the error system G e (s) := G(s) G r (s) = c e (si A e ) 1 b e Let P e and Q e be the error gramians: P e = A e P e + P e A T e + b e b T e = 0, Q e A e + A T e Q e + c T e c e = 0 P 11 P 12 P T 12 P 22, Q e = Q 11 Q 12 Q T 12 Q 22 G e (s) 2 H 2 = c e P e c T e : = First-order necessary conditions: P T 12Q 12 + P 22 Q 22 = 0 Q T 12b + Q 22 b r = 0 c r P 22 cp 12 = 0. 16

17 Equivalently, V = P 12 P 1 22, Z = Q 12Q 1 22 and H 2 Iteration: A r = Z T AV, b r = Z T b, c r = cv. 1. Choose an initial G r (s) = c r (si r A r ) 1 b r. 2. Compute P e and Q e 3. Define V = P 12 P 1 22, Z = Q 12Q Let A r = Z T AV, b r = Z T b, c r = cv. 5. Return to Step 1. Two Lyapunov equations at each step. Similar framework by Hyland and Bernstein [1985] 17

18 Framework of Meier and Luenberger [1967] Let G r (s) = c r (si r A r ) 1 b r solves the optimal H 2 problem Let ˆλ i = λ i (A r ), i.e. the Ritz values. First-order conditions: G( ˆλ i ) = G r ( ˆλ i ), and d ds G(s) = d s= ˆλi ds G r(s) s= ˆλi Match the first two moments at the mirror images of the Ritz values. First-order conditions as interpolation. Rational Krylov Framework 18

19 Theorem: The two frameworks are equivalent. Proof: Starting point for Lyapunov Interpolation Framework: Lemma: (Gallivan et al. [2004], Antoulas/Sorensen [2002]) Let V solves AV + VA T r + bb T r = 0. Then, [ ] Ran(V) = Span ( ˆλ 1 I A) 1 b,, ( ˆλ r I A) 1 b. Starting point for Interpolation Lyapunov Framework: Model reduction via rational Krylov projection. 19

20 For the H 2 problem, simply set σ i = ˆλ i ˆλ i NOT known a priori = Needs iterative rational steps An Iterative Rational Krylov Algorithm (IRKA): (G, Beattie, Antoulas [2004]) 1. Choose σ i for i = 1,..., r. 2. V = Span [ (σ 1 I A) 1 b,, (σ r I A) 1 b ], 3. Z = Span [ (σ 1 I A T ) 1 c T,, (σ r I A T ) 1 c T ], Z T V = I r. 4. while [relative change in σ j ] > ǫ (a) A r = Z T AV, (b) σ i λ i (A r ) for i = 1,..., r (c) V = Span [ (σ 1 I A) 1 b,, (σ r I A) 1 b ]. (d) Z = Span [ (σ 1 I A T ) 1 c T,, (σ r I A T ) 1 c T ], Z T V = I r. 5. A r = Z T AV, b r = Z T b, c r = cv Upon convergence, first-order conditions satisfied via Krylov projection framework, no Lyapunov solvers 20

21 No methods guarantee convergence to global minimum. Question: Global minimum of a restricted H 2 minimization problem? Corollary: (Gaier 1980) Given stable G(s), and the stable reduced poles α 1,...,α r, define Ĝ(s) := β 0 + β 1 s + + β r s r (s α 1 )...(s α r ). Then G(s) Ĝ(s) H 2 is minimized if and only if G(s) = Ĝ(s) for s = α 1, α 2,..., α r. Upon convergence, IRKA minimizes the H 2 norm of the error system among all possible reduced models having the same reduced poles λ j. 21

22 Convergence? Understood better and better every day!!! A fixed point iteration: { } ({ }) (k+1) (k) σ i = f σ i Π (k+1) = h (Π (k)) Usual outcome is (numerical) convergence in 4 5 steps Convergence failure in rare circumstances. Newton Iteration Framework: Jacobian J: Sensitivity of λ i (A r ) wrt {σ i } Requires solving an r r generalized eigenvalue problem {σ i } (k+1) = {σ i } (k) (I + J) 1 ( {σ i } (k) + {λ i (A r )} (k)). 22

23 Stability? A r = Z T AV nonnormal reduced order model Reduced order stability not guaranteed in general. But, very hard to force convergence to unstable model (occasional unstable models can occur at intermediate stages) Fairly robust with respect to initial shift selection. Gugercin [CDC-2005]: Replace Z by QV(V T QV) 1 where implies stability. A T Q + QA + c T c = 0. 23

24 EXTREMELY small order benchmark examples Model r IRKA GFM OPM BTM FOM FOM FOM FOM Divergent FOM FOM Divergent FOM FOM FOM Divergent FOM FOM GFM: Gradient Flow Method of Yan and Lam [1999] OPM: Optimal Projection Method of Hyland and Bernstein [1985] BTM: Balanced Truncation Method of Moore [1981] FOM-1: n = 4, FOM-2: n = 7, FOM-3: n = 4, FOM-4: n = 2, 24

25 ISS 12a Module n = Reduce to r = 2 : 2 : 60 Compare with balanced truncation Comparision between IRKA and BT 1 Evolution of IRKA for r=2 Real(σ 1 ) Relative H 2 error 10 1 BT Imag(σ 1 ) IRKA r: reduced order Relative H 2 error Number of iterations 25

26 Part I: Conclusions and Future Work: Equivalence of first-order conditions for the H 2 problem Iterative Rational Krylov for optimal H 2 reduction First-order conditions while staying in Krylov framework No Lyapunov equations need to be solved Good H performance as well (Zolatorjov Problem (Beattie [2005])). Some open issues remain for convergence and stability. Newton s Iteration Formulation Application to controller reduction: Gugercin/Antoulas/Beattie [2005] Variations that guarantee stability (Gugercin [2005]) Find another name and acronym better than IRKA 26

27 Inexact Solves in Krylov-based Model Reduction Need for more detail and accuracy in the modeling stage System dimension n: O(10 6 ) or more (σi A)v = b cannot be solved directly Inexact solves need to be employed in constructing V and Z Questions: 1. What are the perturbation effects on interpolation? 2. Robustness with respect to the inexact solves? 3. What are the effective preconditioning, restarting strategies? 4. What is the effect on (the optimality of) the reduced model? 27

28 For simplicity, consider the one-sided projection, i.e. V = Z. Let ˆv j be an inexact solution for (σ j I A)v j = b (σ j I A)ˆv j b = δb j with δb j b ǫ Define δv j := ˆv j v j = (σ j I A) 1 δb j, and ˆK := [ (σ 1 I A) 1 b + δv 1, (σ r I A) 1 ] b + δv r. Inexact Krylov-based reduced model obtained by A r = V T A V, b r = V T b, c r = c V, where VT V = Ir. where V is an orthogonal basis for Range( ˆK) 28

29 Theorem: Given the above set-up, c r (σ j I r A r ) 1 b r = c(σ j I n A) 1 b + ε fwd = c(σ j I n A) 1 (b + b j ) where ε fwd = c [ (σ j I n A) 1 V(σ j I r A r ) 1 V T ] δb j. b j = [I n (σ j I n A)V(σ j I r A r ) 1 V T ]δb j. ε fwd : Forward error, b j : Backward error How well V(σ j I r A r ) 1 V T approximates (σ j I n A) 1 Expect optimal model to be robust with respect to inexact solves. Same analysis valid for the two-sided projection as well. 29

30 GMRES: 1. The same Krylov subspace for each (σ j I A)v j = b AW k = W k+1 Hk min σ j Ĩ H k b e 1 2. Span{v j } r j=1 is important, rather than each v j = min x ˆx 1,...,ˆx l (σ l+1 I A)x b 3. Two-sided case: BiCG,... Preconditioning: 1. If σ j is close to σ j+1, can re-use preconditioners for different linear systems 2. Cost of recomputing vs cost of using a close-by preconditioner 30

31 Inexact IRKA (I-IRKA) IRKA requires solving 2r linear systems at each step Expensive if n = O(10 6 ) Recall: {σ j } converge fast Use the solution from the previous step as an initial guess for the next step Expect faster convergence for a fixed tolerance value Optimal reduced model: Expect robustness 31

32 Example: Optimal Cooling of Steel Profiles ( P. Benner) G(s) = c(se A) 1 b, n = 20, 209 Bad shift selection: σ i = logspace( 8, 4, 6) r = 6 via Rational Krylov (RK) and Inexact-RK (I-RK). I-RK uses GMRES with tol = Bode Plots of H(s), H 1 (s) (RK), and H 2 (s) (I IRKA) H(s) H 1 (s) H 2 (s) Error Errors in the computed first moments Relative error Absolute error H(jw) 10 3 H(s) H 1 (s) = H(s) H 2 (s) = H 1 (s) H 2 (s) = Error Errors in the computed second moments 10 0 Absolute error Relative error frequency (rad/sec) σ 32

33 Optimal {σ i } obtained via IRKA Use these {σ i } in I-RK. I-RK uses GMRES with tol = Bode Plots of H(s), H 1 (s) (IRKA), and H 2 (s) (I IRKA) H(s) H 1 (s) H 2 (s) 10 5 Errors in the computed first moments Relative error 10 2 Error Absolute error H(jw) H(s) H 1 (s) = H(s) H 1 (s) = H 1 (s) H 2 (s) = Error Errors in the computed second moments 10 0 Relative error Absolute error frequency (rad/sec) σ 33

34 Same model with n = 79, 841 (Finer discretization) r = 6 via IRKA and I IRKA (tol = ) IRKA: Initial guess from the previous step H(jw) Amplitude Bode plots of H(s), H 1 (s) and H 2 (s) H(s) H 1 (s) H 2 (s) frequency (rad/sec) Number of GMRES steps Total Number of GMRES steps at the k th Iteration Iteration Index H(s) H 1 (s) = H(s) H 2 (s) = , H 1 (s) H 2 (s) =

35 Part II: Conclusions and Future Work n >> 10 6 : Forces usage of Inexact Solves in Krylov-based reduction Perturbation effects: Backward and forward error analysis framework Good/Optimal shift selection robust with respect to inexact solves I-IRKA (Locally) optimal reduced models for n > 10 6 without user intervention Acceleration strategies Open issues: Global H 2 and/or H perturbation effects Modifications to GMRES, effective preconditioning strategies Scalable parallel versions A large-scale easy-to-use model reduction toolbox Modify the algorithms to fit into the framework of, e.g., Trilinos Implementation on Virginia Tech.-System X 35

36 Alexei Nikolaevich Krylov [1931] "On the numerical solution of the equation by which, in technical matters, frequencies of small oscillations of material systems are determined" to compute the characteristic polynomial coefficients.

37 Controller reduction for large-scale systems Consider an n th order plant G(s) = c(si A) 1 b n th κ order stabilizing controller: K(s) = c K (si A K ) 1 b K + d K LQG, H control designs n κ = n (i) Complex hardware (ii) Degraded accuracy (iii) Degraded computational speed Obtain K r (s) of order r n κ to replace K(s) in the closed loop. 37

38 Controller reduction via frequency weighting Small open loop error K(s) K r (s) not enough. Minimize the weighted error: W o (s)(k(s) K r (s))w i (s). How to obtain the weights W o (s) and W i (s)? If K(s) and K r (s) have the same number of unstable poles and if [K(s) Kr (s)]]g(s)[i + G(s)K(s)] 1 < 1, or [I + G(s)K(s)] 1 G(s)[K(s) K r (s)] < 1, = K r (s) stabilizes G(s). = 38

39 For stability considerations: W i (s) = I and W o (s) = [I + G(s)K(s)] 1 G(s) or W o (s) = I and W i (s) = G(s)[I + G(s)K(s)] 1. To preserve closed-loop performance: W i (s) = [I + G(s)K(s)] 1 and W o (s) = [I + G(s)K(s)] 1 G(s). Solved by frequency-weighted balancing (Anderson and Liu [1989], Schelfhout and De Moor [1996], Varga and Anderson [2002] ). Requires solving two Lyapunov equations of order n + n κ. A i P + PA T i + b i b T i = 0, A T o Q + QA o + c T o c o = 0, A i,b i : K(s)W i (s), A o,c o : W o (s)k(s) Balance P and Q. 39

40 Controller-reduction via Krylov Projection How to modify IRKA for the controller reduction problem? Let W i (s) = I and W o (s) = [I + G(s)K(s)] 1 G(s) A K P + PA T K + b K b T K = 0 }{{} unweighted Lyapunov eq. A T wq + QA w + c T wc w = 0. }{{} weighted Lyapunov eq. Z = K(A T,C T, σ i ), and V = K(A,B, µ j ) Z and σ i : Reflect W o (s): the closed-loop information. σ i = jw i over the region where W o (jw) is dominant V and µ j : Obtained in an (optimal) open loop sense. µ j : From an iterative rational Krylov iteration 40

41 An Iterative Rational Krylov Iteration for Controller Reduction: 1. Choose σ i = jw i, for i = 1,..., r where w i is chosen to reflect W o (jw). 2. Z = Span [ (σ 1 I A T K ) 1 c T K (σ ri A T K ) 1 c T K ] with Z T Z = I r. 3. V = Z 4. while [relative change in µ j ] > ǫ (a) A r = Z T A K V, (b) µ j λ i (A r ) for j = 1,..., r (c) V = Span [ (µ 1 I A K ) 1 b K (µ r I A K ) 1 ] b K with Z T V = I r. 5. A r = Z T A K V, b r = Z T b K, c r = c K V Z K r (s) includes the closed loop information V K r (s) is optimal in a restricted H 2 sense Π = ZVT 41

42 International Space Station Module 1R: n = 270. G(s) is lightly damped Long-lasting oscillations. K(s) is designed to remove these oscillations. n κ = x 10 3 Impulse response of the open loop system 6 x 10 3 Impulse response of the closed loop system Amplitude 2 1 Amplitude time (sec) time (sec) Reduce the order to r = 19 using iterative Rational Krylov and to r = 23 using one-sided frequency weighted balancing 42

43 FWBR: Frequency-weighted balancing with W i (s) = I and W o (s) = [I + G(s)K(s)] 1 G(s). IRK-CL: Iterative Rational Krylov - Closed Loop version: σ i reflect the weight W o (s). 30 Bode Plot of W o (s) = T(s) = [ I + G(s)K(s)] 1 G(s) Singular Values (db) Frequency (rad/sec) σ i = j logspace( 1, 2, 10) rad/sec 43

44 Singular Values (db) ISS Example: Bode Plots of reduced closed loop systems T T IRK CL T FW20 T FW23 Relative Errors H error T T FW T T FW23 T T IRK CL Frequency (rad/sec) ISS Model: Error in closed loop systems T T FW20 Relative Errors H 2 error T T FW T T FW23 T T IRK CL Singular Values (db) T T FW23 Weighted Errors H 2 error W i (K K FW20 ) < T T IRK CL W i (K K FW23 ) < 1 W i (K K IRK CL ) < Frequency (rad/sec) 44

45 An Unstable Model: n=2000. K(s) of order n κ = 2000 stabilizes the model. 70 Impulse response of G(s) Amplitude Impulse Response of T(s) Amplitude time (sec) K(s) has four unstable poles. 45

46 Reduce the order to r = 14: Stabilizing controller K r (s) has 4 unstable poles as desired Bode Plots of T(s) and T r (s) T(s) T r (s) 0.5 Impulse Response T(s) T r (s) Singular Values Amplitude Singular Values 10 0 Bode Plots of K(s) and K r (s) 10 1 K(s) K r (s) Amplitude Response to u(t)= sin (4t) T(s) T r (s) Frequency (rad/sec) time (sec) 46