Moving Frontiers in Model Reduction Using Numerical Linear Algebra

Using Numerical Linear Algebra Max-Planck-Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory Group Magdeburg, Germany Technische Universität Chemnitz Fakultät für Mathematik Mathematik in Industrie und Technik Chemnitz, Germany joint work with Tobias Breiten, Jens Saak (TU Chemnitz), Tobias Damm (TU Kaiserslautern) iwasep 8 Berlin, June 28 July 1, 2010

Outline Model 1 Model Order Basic Ideas Large-Scale Linear Systems 2 Short Solving Large-Scale Lyapunov Equations : Bilinear Model Order 3 Model Short : Moment Matching for Bilinear Systems : Moment Matching for Quadratic-Bilinear Approximations 4 2/30

Model Order Basic Ideas Large-Scale Linear Systems Model Dynamical Systems j Σ(p) : with 0 = f (t, x(t), tx(t), ttx(t), u(t), p), x(t 0) = x 0, (a) y(t) = g(t, x(t), tx(t), u(t), p) (b) (generalized) states x(t) x(t; p) X, inputs u(t) U, outputs y(t) y(t; p) Y, (b) is called output equation, p R d is a parameter vector. 3/30

Model Order Basic Ideas Large-Scale Linear Systems Model Dynamical Systems j Σ(p) : 0 = f (t, x(t), tx(t), ttx(t), u(t), p), x(t 0) = x 0, (a) y(t) = g(t, x(t), tx(t), u(t), p) (b) (a) may represent system of ordinary differential equations (ODEs); system of differential-algebraic equations (DAEs); system of partial differential equations (PDEs); a mixture thereof. 3/30

Model Order Basic Ideas Basic Ideas Large-Scale Linear Systems Model Main idea Replace differential equation by low-order one while preserving input-output behavior as well as important system invariants and physical properties! Original System j E(x, p)ẋ = f (t, x, u, p), Σ(p) : y = g(t, x, u, p). states x(t; p) R n, inputs u(t) R m, outputs y(t; p) R q, parameters p R d. Reduced-Order System bσ(p) : j Ê(ˆx, p) ˆx = b f (t, ˆx, u, p), ŷ = bg(t, ˆx, u, p). states ˆx(t; p) R r, r n inputs u(t) R m, outputs ŷ(t; p) R q, parameters p R d. 4/30

Large-Scale Linear Systems Basic Ideas Large-Scale Linear Systems Model Linear, time-invariant (LTI) systems j ẋ(t) = Ax + Bu, A R n n, B R n m, Σ : y(t) = Cx + Du, C R q n, D R q m. (A, B, C, D) is a realization of Σ (nonunique). Laplace transform: state-space frequency domain yields transfer function of Σ: Y (s) = (C(sI n A) 1 B + D) U(s). {z } =:G(s) Goal: find Â Rr r, ˆB R r q, Ĉ Rq r, D R q m such that G Ĝ = (C(sI n A) 1 B + D) (Ĉ(sI r Â) 1 ˆB + ˆD) < tol y ŷ tol u. 5/30

Large-Scale Linear Systems Basic Ideas Large-Scale Linear Systems Model Linear, time-invariant (LTI) systems j ẋ(t) = Ax + Bu, A R n n, B R n m, Σ : y(t) = Cx + Du, C R q n, D R q m. (A, B, C, D) is a realization of Σ (nonunique). Model order reduction by projection Galerkin or Petrov-Galerkin-type projection of state-space onto low-dimensional subspace V along W: assume x VW T x =: x, where range (V ) = V, range (W ) = W, W T V = I r. Then, with ˆx = W T x, we obtain x V ˆx and Â = W T AV, ˆB = W T B, Ĉ = CV, ˆD = D. where V c W T c projects onto V c, the complement of V. 5/30

Short Short Lyapunov Equations Bilinear Model Idea (for simplicity, E = I n ) A system Σ, realized by (A, B, C, D), is called balanced, if solutions P, Q of the Lyapunov equations AP + PA T + BB T = 0, A T Q + QA + C T C = 0, satisfy: P = Q = diag(σ 1,..., σ n ) with σ 1 σ 2... σ n > 0. {σ 1,..., σ n } are the Hankel singular values (HSVs) of Σ. Compute balanced realization of the system via state-space transformation T : (A, B, C, D) (TAT 1, TB, CT 1, D)»» A11 A 12 B1 =, A 21 A 22 B 2, ˆ C 1 C 2, D «(Â, ˆB, Ĉ, ˆD) = (A 11, B 1, C 1, D). 6/30

Short Short Lyapunov Equations Bilinear Model Motivation: HSV are system invariants: they are preserved under T and determine the energy transfer given by the Hankel map H : L 2 (, 0) L 2 (0, ) : u y +. In balanced coordinates... energy transfer from u to y + : E := sup u L 2 (,0] x(0)=x 0 y(t) T y(t) dt 0 0 u(t) T u(t) dt = 1 x 0 2 n σj 2 x0,j 2 j=1 = Truncate states corresponding to small HSVs = analogy to best approximation via SVD, therefore balancing-related methods are sometimes called SVD methods. 6/30

Short Short Lyapunov Equations Bilinear Model Implementation: SR Method 1 Compute (Cholesky) factors of the solutions of the Lyapunov equations, P = S T S, Q = R T R. 2 Compute SVD 3 Set SR T = [ U 1, U 2 ] [ Σ1 Σ 2 ] [ V T 1 V T 2 ]. W = R T V 1 Σ 1/2 1, V = S T U 1 Σ 1/2 1. 4 Reduced model is (W T AV, W T B, CV, D). 6/30

Short Short Lyapunov Equations Bilinear Model Properties: Reduced-order model is stable with HSVs σ 1,..., σ r. Adaptive choice of r via computable error bound: y ŷ 2 (2 n ) u 2. k=r+1 σ k General misconception: complexity O(n 3 ) true for several implementations (e.g., Matlab, SLICOT, MorLAB). But: recent developments in Numerical Linear Algebra yield matrix equation solvers with sparse linear systems complexity! 6/30

Solving Large-Scale Lyapunov Equations Short Lyapunov Equations Bilinear Model General form for A, W = W T R n n given and P R n n unknown: 0 = L(Q) := A T Q + QA + W. In large scale applications from semi-discretized control problems for PDEs, n = 10 3 10 6 (= 10 6 10 12 unknowns!), A has sparse representation (A = M 1 K for FEM), W low-rank with W {BB T, C T C}, where B R n m, m n, C R q n, p n. Standard (Schur decomposition-based) O(n 3 ) methods are not applicable! 7/30

Solving Large-Scale Lyapunov Equations ADI Method for Lyapunov Equations Short Lyapunov Equations Bilinear Model For A R n n stable, B R n m (w n), consider Lyapunov equation AX + XA T = BB T. ADI Iteration: [Wachspress 1988] (A + p k I )X (k 1)/2 = BB T X k 1 (A T p k I ) (A + p k I )X k T = BB T X (k 1)/2 (A T p k I ) with parameters p k C and p k+1 = p k if p k R. For X 0 = 0 and proper choice of p k : lim k X k = X (super)linearly. Re-formulation using X k = Y k Y T k yields iteration for Y k... 8/30

Factored ADI Iteration Lyapunov equation 0 = AX + XA T + BB T. Setting X k = Y k Y T k, some algebraic manipulations = Short Lyapunov Equations Bilinear Model Algorithm [Penzl 97/ 00, Li/White 99/ 02, B. 04, B./Li/Penzl 99/ 08] V 1 p 2Re (p 1)(A + p 1I ) 1 B, Y 1 V 1 FOR j = 2, 3,... q V k Re (pk ) `Vk 1 Re (p k 1 (p ) k + p k 1 )(A + p k I ) 1 V k 1 Y k ˆ Y k 1 V k Y k rrlq(y k, τ) At convergence, Y kmax Yk T max X, where % column compression Y kmax = [ V 1... V kmax ], Vk = C n m. Note: Implementation in real arithmetic possible by combining two steps. 9/30

Factored Galerkin-ADI Iteration Lyapunov equation 0 = AX + XA T + BB T Short Lyapunov Equations Bilinear Model Projection-based methods for Lyapunov equations with A + A T < 0: 1 Compute orthonormal basis range (Z), Z R n r, for subspace Z R n, dim Z = r. 2 Set Â := Z T AZ, ˆB := Z T B. 3 Solve small-size Lyapunov equation Â ˆX + ˆX Â T + ˆB ˆB T = 0. 4 Use X Z ˆX Z T. Examples: Krylov subspace methods, i.e., for m = 1: Z = K(A, B, r) = span{b, AB, A 2 B,..., A r 1 B} [Jaimoukha/Kasenally 94, Jbilou 02 08]. K-PIK [Simoncini 07], Z = K(A, B, r) K(A 1, B, r). 10/30

Factored Galerkin-ADI Iteration Lyapunov equation 0 = AX + XA T + BB T Short Lyapunov Equations Bilinear Model Projection-based methods for Lyapunov equations with A + A T < 0: 1 Compute orthonormal basis range (Z), Z R n r, for subspace Z R n, dim Z = r. 2 Set Â := Z T AZ, ˆB := Z T B. 3 Solve small-size Lyapunov equation Â ˆX + ˆX Â T + ˆB ˆB T = 0. 4 Use X Z ˆX Z T. Examples: ADI subspace [B./R.-C. Li/Truhar 08]: Z = colspan [ V 1,..., V r ]. Note: ADI subspace is rational Krylov subspace [J.-R. Li/White 02]. 10/30

Factored Galerkin-ADI Iteration Numerical example Short Lyapunov Equations Bilinear Model FEM semi-discretized control problem for parabolic PDE: optimal cooling of rail profiles, n = 20, 209, m = 7, p = 6. Good ADI shifts CPU times: 80s (projection every 5th ADI step) vs. 94s (no projection). 11/30 Computations by Jens Saak.

Factored Galerkin-ADI Iteration Numerical example Short Lyapunov Equations Bilinear Model FEM semi-discretized control problem for parabolic PDE: optimal cooling of rail profiles, n = 20, 209, m = 7, p = 6. Bad ADI shifts CPU times: 368s (projection every 5th ADI step) vs. 1207s (no projection). 11/30 Computations by Jens Saak.

Sample applications: VLSI design Short Lyapunov Equations Bilinear Model Application in Microelectronics: VLSI Design was implemented in circuit simulator TITAN (Qimonda AG, Infineon Technologies). TITAN simulation results for industrial circuit: 14,677 resistors, 15,404 capacitors, 14 voltage sources, 4,800 MOSFETs. 14 linear subcircuits of varying order extracted and reduced. Voltage in V 1.4 1.2 1 0.8 0.6 Simulation of a very big circuit TITAN run without reduction BTSR-reduction with C 0.4 0.2 0-0.2 0 1e-08 2e-08 3e-08 4e-08 5e-08 6e-08 7e-08 8e-08 9e-08 1e-07 Time in s [Günzel, Diplomarbeit 08; B. 08] 12/30 Supported by BMBF network SyreNe (includes Qimonda, Infineon, NEC), EU Marie Curie grant O-Moore-Nice! (includes NXP), industry grants.

Sample applications: electro-thermic simulation of integrated circuit (IC) [Source: Evgenii Rudnyi, CADFEM GmbH] Short Lyapunov Equations Bilinear Model Test circuit in Simplorer R with 2 transistors. Conservative thermic sub-system in Simplorer: voltage heat, current heat flux. Original model: n = 270.593, m = p = 2 Computing times (CMESS on Intel Xeon dualcore 3GHz, 1 Thread): Solution of Lyapunov equations: 22min. (420/356 columns in solution factors), Computation of ROMs: 44sec. (r = 20) 49sec. (r = 70). Bode diagram (Matlab on Intel Core i7, 2,67GHz, 12GB): using original system 7.5h, with reduced system < 1min. 13/30

Sample applications: electro-thermic simulation of integrated circuit (IC) [Source: Evgenii Rudnyi, CADFEM GmbH] Short Lyapunov Equations Bilinear Model Original model: n = 270.593, m = p = 2 Computing times (CMESS on Intel Xeon dualcore 3GHz, 1 Thread): Solution of Lyapunov equations: 22min. (420/356 columns in solution factors), Computation of ROMs: 44sec. (r = 20) 49sec. (r = 70). Bode diagram (Matlab on Intel Core i7, 2,67GHz, 12GB): using original system 7.5h, with reduced system < 1min. Bode Diagram (Amplitude) 10 2 Transfer functions of original and reduced systems Hankel Singular Values 10 0 Computed Hankel singular values 10 5 10 1 13/30 σ max (G(jω)) 10 0 10 1 original ROM 20 ROM 30 ROM 40 ROM 50 ROM 60 ROM 70 10 2 10 0 10 2 10 4 ω magnitude 10 10 10 15 10 20 50 100 150 200 250 300 350 index

Sample applications: electro-thermic simulation of integrated circuit (IC) [Source: Evgenii Rudnyi, CADFEM GmbH] Short Lyapunov Equations Bilinear Model Original model: n = 270.593, m = p = 2 Computing times (CMESS on Intel Xeon dualcore 3GHz, 1 Thread): Solution of Lyapunov equations: 22min. (420/356 columns in solution factors), Computation of ROMs: 44sec. (r = 20) 49sec. (r = 70). Bode diagram (Matlab on Intel Core i7, 2,67GHz, 12GB): using original system 7.5h, with reduced system < 1min. Absolute Error σ max (G(jω) G r (jω)) 10 1 absolute model reduction error 10 2 10 3 10 4 10 5 10 6 10 7 ROM 20 ROM 30 ROM 40 ROM 50 ROM 60 ROM 70 Relative Error σ max (G(jω) G r (jω)) / G 10 1 relative model reduction error 10 3 10 5 10 7 ROM 20 ROM 30 ROM 40 ROM 50 ROM 60 ROM 70 13/30 10 8 10 2 10 0 10 2 10 4 ω 10 9 10 2 10 0 10 2 10 4 ω

: Bilinear Model Order for Bilinear Systems [Gray/Mesko 98, Condon/Ivanov 05, B./Damm 09] Short Lyapunov Equations Bilinear Model Bilinear control system of the form ẋ = Ax + arise, e.g., in k N j xu j + Bu, y = Cx, j=1 control of PDEs with mixed boundary conditions, approximation of nonlinear systems using Carleman bilinearization. 14/30

: Bilinear Model Order for Bilinear Systems [Gray/Mesko 98, Condon/Ivanov 05, B./Damm 09] Short Lyapunov Equations Bilinear Model 14/30 Bilinear control system of the form arise, e.g., in ẋ = Ax + k N j xu j + Bu, j=1 y = Cx, control of PDEs with mixed boundary conditions, approximation of nonlinear systems using Carleman bilinearization. The solutions of the generalized Lyapunov equations AP + PA T + kx N j PNj T = BB T, A T Q + QA + j=1 kx j=1 N T j QN j = C T C possess certain properties of the reachability and observability Gramians of linear systems, generalized Hankel singular values can be defined, and model reduction analogous to can be based upon them [Al-Baiyat/Bettaye 93].

: Bilinear Model Order for Bilinear Systems [Gray/Mesko 98, Condon/Ivanov 05, B./Damm 09] Energy functionals [Gray/Mesko, IFAC 1998] E c (x 0 ) = min u 2 u L 2 L 2 (,0) x(,u)=0,x(0,u)=x 0? x T 0 P 1 x 0 Short Lyapunov Equations Bilinear Model E o (x 0 ) = max u L 2 (0, ), u L 2 1 y(, x 0, u) 2 L 2? x T 0 Qx 0 14/30

: Bilinear Model Order Exact unreachability Short Lyapunov Equations Bilinear Model Theorem (k = 1 for simplicity) Let AP + PA T + NPN T + BB T = 0. If P 0 then im P is invariant w.r.t. ẋ = Ax + Nxu + Bu. In particular: ker P is unreachable from 0. Proof: Let v ker P. Then 0 = v T NPN T + BB T v B T v = 0 and PN T v = 0 N T ker P ker P ker B T PA T v = 0, i.e. A T ker P ker P. If x(t) im P = (ker P) for some t, then ẋ(t) T v = x(t) T A T v +u(t)x(t) T N T v {z} ker P {z} ker P +u(t) T B {z} T v = 0 =0 15/30 Hence ẋ(t) im P, implying invariance.

: Bilinear Model Order Exact unreachability Short Lyapunov Equations Bilinear Model Theorem (k = 1 for simplicity) Let AP + PA T + NPN T + BB T = 0. If P 0 then im P is invariant w.r.t. ẋ = Ax + Nxu + Bu. In particular: ker P is unreachable from 0. Consequence: If Px 1 is small, then x 1 should be almost unreachable. How can this be quantified? 15/30

: Bilinear Model Order realization Short Lyapunov Equations Bilinear Model Given factorizations P = LL T, L T QL = UΣ 2 U T, the transformation T = LUΣ 1/2 is balancing: the equivalent system ea = T 1 AT, e Nj = T 1 N j T, e B = T 1 B, e C = CT. satisfies P e = Q e = diag (σ 1,..., σ n). If the small Hankel singular values σ r+1,..., σ n vanish: state negligible!(?) Projection on R r!, (r n).» Partition: T = [T 1, T 2], T 1 S1 =. : S 2 A e(r) = S 1AT 1 N e (r) j = S 1N j T 1 eb (r) = S 1B C e (r) = CT 1 Reduced model: x r = e A (r) x r + mx j=1 en (r) j x r u j + e B (r) u ỹ = e C (r) x r. 16/30

: Bilinear Model Order applied to Nonlinear Systems using Carleman Bilinearization Short Lyapunov Equations Bilinear Model Nonlinear control system (SISO): v(t) = f (v(t)) + g (v(t)) u(t), y(t) = c T v(t), v(0) = 0, f (0) = 0 where f, g : R N R N are nonlinear and analytic in v. f (v) A 1 v + 1 2 A 2v v, g(v) B 0 + B 1 v with B 0 R N, A 1, B 1 R N N, A 2 R N N2, [ ] [ v 1 A1 x =, A = 2 A ] 2, v v 0 A 1 I + I A 1 [ ] [ ] B N = 1 0 B0, B =, C = [ c B 0 I + I B 0 0 0 T 0 ]. 17/30

: Bilinear Model Order Numerical examples Short Lyapunov Equations Bilinear Model Nonlinear RC circuit [Chen/White 00, Bai/Skoogh 06]. Carleman bilinearization bilinear system with n = 2, 550, k = 1. Compare bilinear with Krylov subspace method taken from [Bai/Skoogh 06]. Node 1 Node 2 Node n-1 g g g g Node n i=u(t) g C C C C C 18/30 g(v) = exp(40v) + v 1, u(t) = cos(t)

: Bilinear Model Order Numerical examples Short Lyapunov Equations Bilinear Model Nonlinear RC circuit [Chen/White 00, Bai/Skoogh 06]. Carleman bilinearization bilinear system with n = 2, 550, k = 1. Compare bilinear with Krylov subspace method taken from [Bai/Skoogh 06]. 18/30 u(t) = e t u(t) = (cos 2πt 10 + 1)/2 [B./Damm 09]

Numerical Solution of Bilinear Lyapunov Equations Short Lyapunov Equations Bilinear Model 19/30 Bilinear balanced truncation both require solutions of Bilinear Lyapunov Equation AXE T + EXA T + NXN T + BB T = 0. Naive attempt based on fixed-point iteration X j+1 = N 1 AXE T + EXA T + BB T N T not applicable as N often singular. Current best available method: ADI-preconditioned Krylov subspace methods [Damm 08], using L 1 s L(X ) + P(X ) + BB T = 0, where L : X AXE T + EXA T is the associated Lyapunov operator, P : X NXN T is a positive operator, L 1 s is a preconditioner, obtained by running a fixed (low) number s of ADI steps on the Lyapunov part. Problem: equations beyond n = 1, 000 hardly solvable as no version computing low-rank approximations to X is known yet!

Model Short Model Bilinear Nonlinear Computation of reduced-order model by projection Given a linear (descriptor) system Eẋ = Ax + Bu, y = Cx with transfer function G(s) = C(sE A) 1 B, a reduced-order model is obtained using projection matrices V, W R n r with W T V = I r ( (VW T ) 2 = VW T is projector) by computing Ê = W T EV, Â = W T AV, ˆB = W T B, Ĉ = CV. Petrov-Galerkin-type (two-sided) projection: W V, Galerkin-type (one-sided) projection: W = V. 20/30

Model Short Model Bilinear Nonlinear Theorem (simplified) [Grimme 97, Villemagne/Skelton 87] If then span { (s 1 E A) 1 B,..., (s k E A) 1 B } Ran(V ), span { (s 1 E A) T C T,..., (s k E A) T C T } Ran(W ), G(s j ) = Ĝ(s j ), d ds G(s j) = d ds Ĝ(s j), for j = 1,..., k. Remarks: computation of V, W from rational Krylov subspaces, e.g., dual rational Arnoldi/Lanczos [Grimme 97], Iterative Rational Krylov-Algo. [Antoulas/Beattie/Gugercin 07]. 20/30

Model Short Model Bilinear Nonlinear Theorem (simplified) [Grimme 97, Villemagne/Skelton 87] If then span { (s 1 E A) 1 B,..., (s k E A) 1 B } Ran(V ), span { (s 1 E A) T C T,..., (s k E A) T C T } Ran(W ), G(s j ) = Ĝ(s j ), d ds G(s j) = d ds Ĝ(s j), for j = 1,..., k. Remarks: using Galerkin/one-sided projection yields G(s j ) = Ĝ(s j), but in general d ds G(s j) d ds Ĝ(s j). 20/30

Model Short Model Bilinear Nonlinear Theorem (simplified) [Grimme 97, Villemagne/Skelton 87] If then span { (s 1 E A) 1 B,..., (s k E A) 1 B } Ran(V ), span { (s 1 E A) T C T,..., (s k E A) T C T } Ran(W ), G(s j ) = Ĝ(s j ), d ds G(s j) = d ds Ĝ(s j), for j = 1,..., k. Remarks: k = 1, standard Krylov subspace(s) of dimension K moment-matching methods/padé approximation, d i ds G(s1) = d i Ĝ(s i dsi 1 ), i = 0,..., K 1(+K). 20/30

: Moment Matching for Bilinear Systems Input-output characterization of bilinear systems Model Bilinear Nonlinear Recall: bilinear system ẋ = Ax + Nxu + Bu, y = Cx, For I/O-behavior, generalize concepts for linear systems by Volterra series t t1 tk 1 y(t) =... h(t 1, t 2,..., t k )u(t t 1 t k ) 0 k=1 0 0 h(t 1, t 2,..., t k ) = c T e At k N e At2 Ne At1 b degree-k kernel u(t t k )dt k... dt 1 multivariable Laplace transform 21/30 H(s 1, s 2,..., s k ) = c T (s k I A) 1 N (s 2 I A) 1 N(s 1 I A) 1 b k-th transfer function

: Moment Matching for Bilinear Systems High frequency multimoments Model Bilinear Nonlinear s i = ξ 1 i : H(s 1,..., s k ) = c T (s k I A) 1 N (s 2 I A) 1 N(s 1 I A) 1 b = c T (ξ 1 k I A) 1 N (ξ 1 2 I A) 1 N(ξ 1 1 I A) 1 b = c T ξ k (I ξ k A) 1 N ξ 2 (I ξ 2 A) 1 Nξ 1 (I ξ 1 A) 1 b for ξ i 0 (s i ) use Neumann expansion: (I ξ i A) 1 = l i =0 ξ l i i Al i H(s 1,..., s k ) =... m(l 1,..., l k )s l1 1 s l k k l k =1 l 1=1 m(l 1,..., l k ) = c T A l k 1 N A l2 1 NA l1 1 b 22/30 high frequency multimoments

: Moment Matching for Bilinear Systems High frequency multimoments Model Bilinear Nonlinear s i = ξ 1 i : H(s 1,..., s k ) = c T (s k I A) 1 N (s 2 I A) 1 N(s 1 I A) 1 b = c T (ξ 1 k I A) 1 N (ξ 1 2 I A) 1 N(ξ 1 1 I A) 1 b = c T ξ k (I ξ k A) 1 N ξ 2 (I ξ 2 A) 1 Nξ 1 (I ξ 1 A) 1 b for ξ i 0 (s i ) use Neumann expansion: (I ξ i A) 1 = l i =0 ξ l i i Al i m(l 1 ) = c T A l1 1 b Markov parameters m(l 1, l 2 ) = c T A l2 1 NA l1 1 b m(l 1, l 2, l 3 ) = c T A l3 1 NA l2 1 NA l1 1 b 22/30.

: Moment Matching for Bilinear Systems Arbitrary expansion points Model Bilinear Nonlinear Similar for s i σ i C : H(s 1,..., s k ) =... m(l 1,..., l k )s l1 1 1 s l k 1 k l k =1 l 1=1 m(l 1,..., l k ) = ( 1) k c T (A σ k I ) l k N (A σ 2 I ) l2 N(A σ 1 I ) l1 b special case σ i = 0 : m(l 1,..., l k ) = ( 1) k c T A l k N A l2 NA l1 b low frequency multimoments 23/30

: Moment Matching for Bilinear Systems Model reduction Ingredients Model Bilinear Nonlinear Matching multi-moments: multimoments locally characterize input-output behaviour construct reduced system Σ that matches q k multimoments of the first r subsystems of the original system m(l 1,..., l k )! = ˆm(l 1,..., l k ), k = 1,..., r, l j = 1,..., q Construct reduced system by Petrov-Galerkin projection: 8 ˆx(t) = W T {z AV } ˆx(t) + W T {z NV } ˆx(t)u(t) + W {z T b } u(t), >< Â ˆN ˆb ˆΣ : ŷ(t) = c >: {z} T V ˆx(t), x(t) V ˆx(t) ĉ T with V, W R n k, W T V = I. 24/30 Use sequence of nested Krylov subspaces n o K q(a, b) = span b, Ab,..., A q 1 b, A R n n, b R n

: Moment Matching for Bilinear Systems One-sided methods: high frequency multimoments Model Bilinear Nonlinear Theorem Let a bilinear SISO system Σ be given. span{v (1) } = K q (A, b), span{v (k) } = K q (A, NV (k 1) ), k = 2,..., r span{v } = span { r k=1 span{v (k) } } W arbitrary left inverse of V m(l 1,..., l k ) = ˆm(l 1,..., l k ), k = 1,..., r, l j = 1,..., q Example: V (1) = K 10 (A, b), V (2) = K 4 (A, NV (1) [4] ) c T A l1 1 b = ĉ T Â l1 1ˆb, l 1 = 1,..., 10 c T A l2 1 NA l1 1 b = ĉ T Â l2 1 ˆNÂ l1 1ˆb, l1, l 2 = 1,..., 4 25/30

: Moment Matching for Bilinear Systems One-sided methods: arbitrary multimoments Model Bilinear Nonlinear Multimoment-matching for different expansion points to cover broader frequency range: Theorem Let a bilinear SISO system Σ be given. span{v (1) } = K q((a σ 1I ) 1, (A σ 1I ) 1 b), span{v (k) } = K q((a σ k I ) 1, (A σ k I ) 1 NV (k 1) ), n Sr o span{v } = span k=1 span{v (k) } W arbitrary left inverse of V m(l 1,..., l k ) = ˆm(l 1,..., l k ), k = 1,..., r, l j = 1,..., q Special cases: V T V = I, W T = V T orthogonal projection first approach, proposed by [Phillips 03], see also [B./Feng 07] for multi-moment matching proof. 26/30

: Moment Matching for Bilinear Systems One-sided methods: arbitrary multimoments Model Bilinear Nonlinear Multimoment-matching for different expansion points to cover broader frequency range: Theorem Let a bilinear SISO system Σ be given. span{v (1) } = K q((a σ 1I ) 1, (A σ 1I ) 1 b), span{v (k) } = K q((a σ k I ) 1, (A σ k I ) 1 NV (k 1) ), n Sr o span{v } = span k=1 span{v (k) } W arbitrary left inverse of V m(l 1,..., l k ) = ˆm(l 1,..., l k ), k = 1,..., r, l j = 1,..., q Special cases: V T V = I, W T = (V T A 1 V ) 1 V T A 1 multiply state equation by A 1, proposed by [Skoogh/Bai 06] seems to yield better results for bilinearized systems. 26/30

: Moment Matching for Bilinear Systems Two-sided methods Model Bilinear Nonlinear Better choices for projection matrix W? span{w (1) } = K q (A T, c), span{w (k) } = K q (A T, N T W (k 1) ), k = 2,..., r span{w } = span { r k=1 span{w (k) } } V (1) = K 6 (A, b), W (1) = K 6 (A T, c) m(l 1 ) = ˆm(l 1 ), l 1 = 1,..., 12, m(l 1, l 2 ) = ˆm(l 1, l 2 ), l 1, l 2 = 1,..., 6 significantly more multimoments are preserved. Number of matched subsystems automatically doubles. 27/30

: Moment Matching for Bilinear Systems Numerical examples: nonlinear RC circuit Node 1 Node 2 Node n-1 Node n g g g g i=u(t) g C C C C C Model Bilinear Nonlinear v(t) : node voltages v 1 (t),..., v N (t), N = 50 dim Σ = 2550 u(t) : independent current source, C = 1, g(v) = exp(40v) + v 1 y(t) : voltage between node 1 and ground 28/30

: Moment Matching for Bilinear Systems Numerical examples: nonlinear RC circuit Model Bilinear Nonlinear 28/30 Projection subspaces: High frequency multimoments ( ): V (1) = K 19 (A, b), V (2) = K 4 (A, NV (1) [4] ) V = V (1) V (2), V T V = I Low frequency multimoments (σ j = 0): V (1) = K 19 (A 1, A 1 b), V (2) = K 4 (A 1, A 1 NV (1) [4] ) V = V (1) V (2), V T V = I Multiple interpolation points (σ j = 0, 1, 10, 100, ): e.g. σ j = 10: V (1) = K q1 ((A 10 I ) 1, (A 10 I ) 1 b) V (2) = K q2 ((A 10 I ) 1, (A 10 I ) 1 NV (1) [p] ) First and second order multimoments are preserved.

: Moment Matching for Bilinear Systems Numerical examples: nonlinear RC circuit Simulation results: Model Bilinear Nonlinear 28/30

Model Bilinear Nonlinear : Moment Matching for Quadratic-Bilinear Approximations Many nonlinear dynamics can be modeled by quadratic bilinear differential algebraic equations (QBDAEs), i.e. Eẋ = A 1 x + A 2 x x + Nxu + bu, y = cx, where E, A 1, N R n n, A 2 R n n2, b, c T R n. Combination of quadratic and bilinear control systems. Variational analysis allows characterization of input-output behavior via generalized transfer functions, e.g. H 1(s) = c (se A 1) 1 b, {z } G(s) H 2(s 1, s 2) = 1 2 c ((s1 + s2) E A1) 1 [A 2(G(s 1) G(s 2) + G(s 2) G(s 1)) +N (G(s 1) + G(s 2))] 29/30

: Moment Matching for Quadratic-Bilinear Approximations Model Bilinear Nonlinear Which systems can be transformed? Theorem [Gu 09] Assume that the state equation of a nonlinear system Σ is given by ẋ = a 0 x + a 1 g 1 (x) +... + a k g k (x) + bu, where g i (x) : R n R n are compositions of rational, exponential, logarithmic, trigonometric or root functions, respectively. Then Σ can be transformed into a quadratic bilinear differential algebraic equation of dimension N > n. transformation is not unique original system has to be increased before reduction is possible minimal dimension N? 29/30

: Moment Matching for Quadratic-Bilinear Approximations Model Bilinear Nonlinear Example Consider the following two dimensional nonlinear control system: q ẋ 1 = exp( x 2) x1 2 + 1, ẋ 2 = sin x 2 + u. Introduce useful new state variables, e.g. q x 3 := exp( x 2), x 4 := x1 2 + 1, x5 := sin x2, x6 := cos x2. System can be replaced by a QBDAE of dimension 6: ẋ 1 = x 3 x 4, ẋ 2 = x 5 + u, ẋ 3 = x 3 (x 5 + u), ẋ 5 = x 6 (x 5 + u), 2 x1 x3 x4 ẋ 4 =, 2 x 4 ẋ 6 = x 5 (x 5 + u). 29/30

: Moment Matching for Quadratic-Bilinear Approximations Model Bilinear Nonlinear Multi-moment-Matching for QBDAEs Construct reduced order model by projection: Ê = Z T EZ, Â 1 = Z T A 1Z, ˆN = Z T NZ, Â 2 = Z T A 2Z Z, ˆb = Z T b, ĉ = cz Approximate values and derivatives ( multi-moments ) of transfer functions about an expansion point σ using Krylov spaces, e.g. span{v } = K 6 (A σe, A σb) span{w 1} = K 3 (A 2σE, A 2σ(A 2V 1 V 1 N 1V 1)) span{w 2} = K 2 (A 2σE, A 2σ(A 2(V 2 V 1 + V 1 V 2) N 1V 2)) span{w 3} = K 1 (A 2σE, A 2σ(A 2(V 2 V 2 + V 2 V 2))) span{w 4} = K 1 (A 2σE, A 2σ(A 2(V 3 V 1 + V 1 V 3) N 1V 3)), 29/30 with A σ = (A 1 σe) 1 and V i denoting the i-th column of V derivatives match up to order 5 (H 1) and 2 (H 2), respectively.

: Moment Matching for Quadratic-Bilinear Approximations Model Bilinear Nonlinear Numerical Example FitzHugh-Nagumo system: simple model for neuron (de-)activation. ɛv t(x, t) = ɛ 2 v xx(x, t) + f (v(x, t)) w(x, t) + g, w t(x, t) = hv(x, t) γw(x, t) + g, with f (v) = v(v 0.1)(1 v) and initial and boundary conditions v(x, 0) = 0, w(x, 0) = 0, x [0, 1] v x(0, t) = i 0(t), v x(1, t) = 0, t 0, where ɛ = 0.015, h = 0.5, γ = 2, g = 0.05, i 0(t) = 50000t 3 exp( 15t) parameter g handled as an additional input original state dimension n = 2 400, QBDAE dimension N = 3 400, reduced QBDAE dimension r = 26, chosen expansion point σ = 1 [B./Breiten 2010] 29/30

: Moment Matching for Quadratic-Bilinear Approximations Numerical Example Model Bilinear Nonlinear 2d Phase Space [B./Breiten 2010] 29/30

: Moment Matching for Quadratic-Bilinear Approximations Numerical Example Model Bilinear Nonlinear 3d Phase Space [B./Breiten 2010] 29/30

Model 30/30 Model reduction for nonlinear systems based on Carleman bilinearization and bilinear, QBDAE transformation and multi-moment matching has high potential for many classes of nonlinear dynamical systems. Current work: High dimensions can be dealt with using tensor product structures of coefficient matrices already done for bilinear Krylov subspaces [Condon/Ivanov 07], for Gramian computation in progress [B./Damm]. QBDAE is exact for many nonlinearities, e.g. + reaction-diffusion systems and population balances; + various PDEs with nonlinear convective terms x. x: Burgers, Euler, Navier-Stokes, Kuramoto-Sivashinsky eqns; hence, reduced-order model will have the same nonlinear structure. Enhance efficiency of QBDAE approach using tensor decomposition, low-rank and sparse approximations.

Model reduction for nonlinear systems based on Carleman bilinearization and bilinear, QBDAE transformation and multi-moment matching has high potential for many classes of nonlinear dynamical systems. Model Thank you for your attention! 30/30