The rational Krylov subspace for parameter dependent systems. V. Simoncini

The rational Krylov subspace for parameter dependent systems V. Simoncini Dipartimento di Matematica, Università di Bologna valeria.simoncini@unibo.it 1

Given the continuous-time system Motivation. Model Order Reduction Σ = A B C, A R n n, B,C T tall Analyse the construction of a reduced system Σ = Ã B, Ã of size m n C so that all relevant properties are captured by Σ 2

Motivation (cont d): Linear Dynamical Systems Time-invariant linear system: x (t) = Ax(t)+Bu(t), x(0) = x 0 y(t) = Cx(t) Approximation of parameter-dependent systems Approximation of the matrix Transfer function Other problems (e.g., Matrix equations) Emphasis: A large dimensions, W(A) C 3

Projection methods: the general idea Given space K R n of size O(m) and (orthonormal) basis V m, A A m = V T mav m B B m = V T mb C C m = CV m Σ = A m B m C m 4

Projection methods: the space Choices of K in the literature: Standard Krylov subspace: K m (A,B) = Range{[B,AB,...,A m 1 B]} Shift-Invert Krylov subspace: K m ((A σi) 1,B) = span{b,(a σi) 1 B,...,(A σi) (m 1) B}; often σ = 0 Extended Krylov subspace: EK m (A,B) = K m (A,B)+K m (A 1,A 1 B) Rational Krylov subspace: K m (A,B,s) = span{(a s 1 I) 1 B,(A s 2 I) 1 B,...,(A s m I) 1 B} in the past, s = [s 1,...,s m ] a-priori 5

Projection methods: the space Choices of K in the literature: Standard Krylov subspace: K m (A,B) = Range{[B,AB,...,A m 1 B]} Shift-Invert Krylov subspace: K m ((A σi) 1,B) = span{b,(a σi) 1 B,...,(A σi) (m 1) B}; often σ = 0 Extended Krylov subspace: EK m (A,B) = K m (A,B)+K m (A 1,A 1 B) Rational Krylov subspace: K m (A,B,s) = Range{[(A s 1 I) 1 B,(A s 2 I) 1 B,...,(A s m I) 1 B]} usually s = [s 1,...,s m ] a-priori 6

Projection methods: the space Choices of K in the literature: Standard Krylov subspace: K m (A,B) = Range{[B,AB,...,A m 1 B]} Shift-Invert Krylov subspace: K m ((A σi) 1,B) = Range{[B,(A σi) 1 B,...,(A σi) (m 1) B]}; often σ = 0 Extended Krylov subspace: EK m (A,B) = K m (A,B)+K m (A 1,A 1 B) Rational Krylov subspace: K m (A,B,s) = Range{[(A s 1 I) 1 B,(A s 2 I) 1 B,...,(A s m I) 1 B]} usually s = [s 1,...,s m ] a-priori 7

Projection methods: the space Choices of K in the literature: Standard Krylov subspace: K m (A,B) = Range{[B,AB,...,A m 1 B]} Shift-Invert Krylov subspace: K m ((A σi) 1,B) = Range{[B,(A σi) 1 B,...,(A σi) (m 1) B]}; often σ = 0 Extended Krylov subspace: EK m (A,B) = K m (A,B)+K m (A 1,A 1 B) Rational Krylov subspace: K m (A,B,s) = Range{[(A s 1 I) 1 B,(A s 2 I) 1 B,...,(A s m I) 1 B]} usually s = [s 1,...,s m ] a-priori 8

Projection methods: the space Choices of K in the literature: Standard Krylov subspace: K m (A,B) = Range{[B,AB,...,A m 1 B]} Shift-Invert Krylov subspace: K m ((A σi) 1,B) = Range{[B,(A σi) 1 B,...,(A σi) (m 1) B]}; often σ = 0 Extended Krylov subspace: EK m (A,B) = K m (A,B)+K m (A 1,A 1 B) Rational Krylov subspace: K m (A,B,s) = Range{[(A s 1 I) 1 B,(A s 2 I) 1 B,...,(A s m I) 1 B]} in the past, s = [s 1,...,s m ] a-priori 9

Parameter-dependent linear systems x = f σ (A)b x K Here, f σ (λ) = (λ+σ) 1. A different perspective: Shifted linear systems. Many shifts in a wide range (e.g., Structural dynamics, electromagn.) (A+σ j I)x = v, σ j [α,β], large interval,j = O(100) Few (possibly complex) shifts (e.g., quadrature formulas) z = k ω j (A σ j I) 1 v j=1 Transfer function h(σ) = c (A iσi) 1 b, σ [α,β] 10

Parameter-dependent linear systems x = f σ (A)b x K Here, f σ (λ) = (λ+σ) 1. A different perspective: Shifted linear systems. Many shifts in a wide range (e.g., Structural dynamics, electromagn.) (A+σ j I)x = v, σ j [α,β], large interval,j = O(100) Few (possibly complex) shifts (e.g., quadrature formulas) z = k ω j (A σ j I) 1 v j=1 Transfer function h(σ) = c T (A iσi) 1 b, σ [α,β] 11

Shifted systems Approximation. Defining T m = V mav m, x x m = V m f σ (T m )e 1 = V m (T m +σi) 1 e 1 Galerkin condition: r m := b (A+σI)x m K. (standard Galerkin-type approximation for shifted systems, cf. FOM, CG,...) Key fact: A single K for all shifted systems. Note: Solve systems with A to approximate (A+σI) 1 v Added feature: restarting made easy 12

A numerical example Matrix A discretization of L(u) = u+50(x+y)(u x +u y ) b = 1, 500 shift in [0,5] ( backslash takes 2.52 secs for each system of size 160 000) n subspace std Krylov Extended Krylov dimension CPU time (#cycles) CPU time (#cycles) 2500 10 1.50 (41) 0.65 (4) 2500 20 1.57 (14) 0.63 (2) 10 000 10 3.23 (49) 1.96 (7) 10 000 20 4.35 (27) 1.75 (3) 160 000 10 399.80 (>300) 62.81 (13) 160 000 20 356.63 (97) 109.87 (6) 13

Sylvester-type equations In shifted systems, the right-hand side may depend on the parameter: B = [b(σ 1 ),...,b(σ s )] AX +XS = B, S = diag(σ 1,...,σ s ) (special case of more general Sylvester equation) Generate approximation space starting with B = V 1 β 0 (V 1 orthnormal columns, β 0 possibly rectangular) 14

A numerical example. Direct frequency analysis in structural dynamics (K σ 2 M)x(σ) = b(σ), size 3627 K complex symmetric, B = [b(σ 1 ),...,b(σ s )] of rank 2. Simultaneous solution: K X +MXS = B, S = diag(σ1,...,σ 2 s). 2 For M real sym nonsingular, M = LL T and (L 1 K L T ) X + XS = L 1 B, X = UX with L 1 K L T complex symmetric bilinear form x T y 15

A numerical example. Direct frequency analysis in structural dynamics Comparison with: Q(σ)x(σ) = b(σ), Q(σ) = K σ 2 M for each σ, (complex symmetric CG method, preconditioned with LDL with drop. tol. 10 2 ) Sparse Direct Complex sym Extended Krylov Extended Krylov #σ Solver CG Direct solves K Iter.solves K 50 12.57 48.30 (108) 1.59 (36) 8.00 (36) 100 24.88 96.33 (108) 2.12 (40) 9.69 (40) 16

Transfer function approximation A with field of values in C h(ω) = c(a iωi) 1 b, ω [α,β] Given space K of size m and V m s.t. K=range(V m ), h(ω) cv m (V T mav m iωi) 1 (V T mb) Next: Classical benchmark experiment with Standard, Shift-invert and Extended Krylov 17

An example: CD Player, n = 120 h(ω) = C 2,: (A iωi) 1 B :,1 m = 20 m = 50 10 4 true 10 2 true 10 2 Krylov extended invert Krylov 10 1 10 0 Krylov extended invert Krylov 10 0 10 1 10 2 10 2 10 3 10 4 10 4 10 5 10 6 10 6 10 7 10 8 10 1 10 0 10 1 10 2 10 3 10 4 10 5 10 6 10 8 10 1 10 0 10 1 10 2 10 3 10 4 10 5 10 6 18

Rational Krylov Subspace Method. Choice of poles K m (A,B,s) = Range{[(A s 1 I) 1 B,(A s 2 I) 1 B,...,(A s m I) 1 B]} cf. General discussion in Antoulas, 2005. Various attempts: Gallivan, Grimme, Van Dooren (1996, ad-hoc poles) Penzl (1999-2000, ADI shifts - preprocessing, Ritz values)... Sabino (2006 - tuning within preprocessing) IRKA Gugercin, Antoulas, Beattie (2008) 19

A new adaptive choice of poles for RKSM K m (A,b,s) = span{(a s 1 I) 1 b,(a s 2 I) 1 b,...,(a s m I) 1 b} s = [s 1,...,s m ] to be chosen sequentially (no fixed m) The fundamental idea: Assume you wish to solve (A si)x = b with a Galerkin procedure in K m (A,b,s). Let V m be orth. basis. The residual satisfies: b (A si)x m = r m(a)b m r m (s), r z λ j m(z) = z s j with λ j = eigs(v T mav m ). Moreover, r m (A)b = min θ 1,...,θ m j=1 m (A θ j I)(A s j I) 1 b j=1 20

A new adaptive choice of poles for RKSM. Cont d r m (z) = m j=1 z λ j z s j, λ j = eigs(v mav m ) For A symmetric: ( s m+1 := arg max s [ λ max, λ min ] ) 1 r m (s) [λ min,λ max ] spec(a) (Druskin, Lieberman, Zaslavski (SISC 2010)) For A nonsymmetric: ( ) 1 s m+1 := arg max s S m r m (s) where S m C + approximately encloses the eigenvalues of A 21

Example of S m. CD Player, m = 12 S m : encloses mirrored current Ritz values: -eigs(v T mav m ) and initial estimates: s (0) 1 = 0.1, s (0) 2,3 = 900±i5 104 5 x 104 4 3 2 imaginary part 1 0 1 2 3 4 5 0 100 200 300 400 500 600 700 800 900 real part poles + -eigs(a) S m 22

Transfer function evaluation. FOM Data set 10 2 10 0 10 1 10 2 10 4 h(ω) 10 0 true Extended Krylov std Krylov invert Krylov rational Krylov h(ω) h approx (ω) 10 6 10 8 10 10 10 12 10 14 Extended Krylov std Krylov invert Krylov rational Krylov 10 1 10 1 10 0 10 1 10 2 10 3 10 4 frequency ω 10 16 10 1 10 0 10 1 10 2 10 3 10 4 frequency ω m = 20. A of size 1006 (normal matrix), s (1) 0 = 1, s (2) 0 = 1000 23

Transfer function evaluation. Flow Data set. h(ω) = c(a iωe) 1 b h(ω) h(ω) h approx (ω) 10 0 10 0 10 2 10 2 10 4 10 4 extended 10 6 10 6 invert arnoldi rational krylov 10 8 10 10 true extended invert arnoldi rational krylov 10 8 10 10 10 12 10 1 10 0 10 1 10 2 10 3 10 4 10 12 10 1 10 0 10 1 10 2 10 3 10 4 m = 20. A,E of size 9669 (nonsym) s (1) 0 = A /(condest(a) E F ), s (1) 0 arg(max R(eigs(A))) (all real poles) 24

Understanding the Rational Krylov subspace. State of the art. Shift selection. - Historically, a-priori selection (possibly with pre-processing) - Given A symmetric, asymptotically optimal rational space for ir by using an equidistributed nested sequence of real shifts (from classical Zolotaryov sol n) Druskin, Knizhnerman, Zaslavsky (SISC 2009) - For f(a)b, variants of adaptive selection (Güttel-Knizhnerman, tr 2012) Convergence Analysis. Mostly very recent, based on classical potential theory. (Beckermann, Druskin, Eiermann, Ernst, Güttel, Knizhnerman, Lieberman, Reichel, Simoncini, Vandebril, Zaslavsky) 25

Projection methods for matrix equations F(X) = 0 linear or quadratic equations, such as: AX +XA T +BB T = 0 AX +XB +C = 0 Lyapunov equation Sylvester equation AX +XA T XBB T X +C T C = 0 Riccati equation With an approximation in the space K with basis V m V m, (Use the Kronecker formulation for the derivation only) residual K VmF(X T m )V m = 0 where X m is the matrix approximation 26

The Lyapunov matrix equation AX +XA T +BB T = 0 Approximation by projection: K of dim. m, V m orthonormal basis. X X m = V m YV T m (V T mav m )Y +Y(V T ma T V m )+V T mbb T V m = 0 With K being the Krylov subspace (Saad, 90, Jaimoukha & Kasenally, 94), the Extended Krylov subspace (Simoncini, 07), the adaptive Rational Krylov subspace (Druskin & Simoncini, 11) 27

Long term Competitor: The Lyapunov equation. Cont d. Alternating Directions Implicit iteration (ADI) (large number of contributions, see, e.g., Benner, Penzl, etc.) problem: computation of parameters Extended Krylov Subspace method numerically outperforms ADI in general Rational Krylov Subspace is at least as good as ADI (Druskin, Knizhnerman, Simoncini, 11, Beckermann 11, theoretical arguments) Convergence Analysis Much is now known on convergence (in the last 5 years!) 28

Conclusions and outlook Projection-type methods still have great potential Second generation Krylov spaces can attack harder problem than standard linear systems Quadratic problems (Heyouni, Jbilou, 09). New very promising results for Riccati eqn (Simoncini, Szyld) www.dm.unibo.it/~simoncin valeria.simoncini@unibo.it References: V. Simoncini, A new iterative method for solving large-scale Lyapunov matrix equations. SIAM J. Scient. Comput., v.29, n.3 (2007), pp. 1268-1288 V. Simoncini, The Extended Krylov subspace for parameter dependent systems. Applied Num. Math. v.60 n.5 (2010) 550-560. V. Druskin and V. Simoncini, Adaptive rational Krylov subspaces for large-scale dynamical systems. Systems and Control Letters, 2011. 29