Model Reduction for Dynamical Systems

Transcription

1 Otto-von-Guericke Universität Magdeburg Faculty of Mathematics Summer term 2015 Model Reduction for Dynamical Systems Lecture 8 Peter Benner Lihong Feng Max Planck Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory Magdeburg, Germany benner@mpi-magdeburg.mpg.de feng@mpi-magdeburg.mpg.de ss15

2 Outline 1 Introduction 2 Mathematical Basics 3 Model Reduction by Projection 4 Modal Truncation 5 Balanced Truncation 6 Moment-Matching 7 Solving Large-Scale Matrix Equations Linear Matrix Equations Numerical Methods for Solving Lyapunov Equations Solving Large-Scale Algebraic Riccati Equations Software Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 2/39

3 Solving Large-Scale Matrix Equations Large-Scale Algebraic Lyapunov and Riccati Equations Algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. Standard (eigenproblem-based) O(n 3 ) methods are not applicable! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 3/39

4 Solving Large-Scale Matrix Equations Large-Scale Algebraic Lyapunov and Riccati Equations Algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. Standard (eigenproblem-based) O(n 3 ) methods are not applicable! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 3/39

5 Solving Large-Scale Matrix Equations Large-Scale Algebraic Lyapunov and Riccati Equations Algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. Standard (eigenproblem-based) O(n 3 ) methods are not applicable! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 3/39

6 Solving Large-Scale Matrix Equations Large-Scale Algebraic Lyapunov and Riccati Equations Algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. Standard (eigenproblem-based) O(n 3 ) methods are not applicable! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 3/39

7 Solving Large-Scale Matrix Equations Large-Scale Algebraic Lyapunov and Riccati Equations Algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. Standard (eigenproblem-based) O(n 3 ) methods are not applicable! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 3/39

8 Solving Large-Scale Matrix Equations Large-Scale Algebraic Lyapunov and Riccati Equations Algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. Standard (eigenproblem-based) O(n 3 ) methods are not applicable! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 3/39

9 Solving Large-Scale Matrix Equations Low-Rank Approximation Consider spectrum of ARE solution (analogous for Lyapunov equations). Example: Linear 1D heat equation with point control, Ω = [ 0, 1 ], FEM discretization using linear B-splines, h = 1/100 = n = 101. Idea: X = X T 0 = X = ZZ T = n λ k z k zk T Z (r) (Z (r) ) T = k=1 r λ k z k zk T. k=1 = Goal: compute Z (r) R n r directly w/o ever forming X! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 4/39

10 Solving Large-Scale Matrix Equations Low-Rank Approximation Consider spectrum of ARE solution (analogous for Lyapunov equations). Example: Linear 1D heat equation with point control, Ω = [ 0, 1 ], FEM discretization using linear B-splines, h = 1/100 = n = 101. Idea: X = X T 0 = X = ZZ T = n λ k z k zk T Z (r) (Z (r) ) T = k=1 r λ k z k zk T. k=1 = Goal: compute Z (r) R n r directly w/o ever forming X! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 4/39

11 Solving Large-Scale Matrix Equations Linear Matrix Equations Equations without symmetry Sylvester equation AX + XB = W discrete Sylvester equation AXB X = W with data A R n n, B R m m, W R n m and unknown X R n m. Equations with symmetry Lyapunov equation AX + XA T = W Stein equation (discrete Lyapunov equation) AXA T X = W with data A R n n, W = W T R n n and unknown X R n n. Here: focus on (Sylvester and) Lyapunov equations; analogous results and methods for discrete versions exist. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 5/39

12 Solving Large-Scale Matrix Equations Linear Matrix Equations Equations without symmetry Sylvester equation AX + XB = W discrete Sylvester equation AXB X = W with data A R n n, B R m m, W R n m and unknown X R n m. Equations with symmetry Lyapunov equation AX + XA T = W Stein equation (discrete Lyapunov equation) AXA T X = W with data A R n n, W = W T R n n and unknown X R n n. Here: focus on (Sylvester and) Lyapunov equations; analogous results and methods for discrete versions exist. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 5/39

13 Linear Matrix Equations Solvability Using the Kronecker (tensor) product, AX + XB = W is equivalent to ( (Im A) + ( B T I n )) vec (X ) = vec (W ). Hence, Sylvester equation has a unique solution M := (I m A) + ( ) B T I n is invertible. 0 Λ (M) = Λ ((I m A) + (B T I n)) = {λ j + µ k, λ j Λ (A), µ k Λ (B)}. Λ (A) Λ ( B) = Corollary A, B Hurwitz = Sylvester equation has unique solution. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 6/39

14 Linear Matrix Equations Solvability Using the Kronecker (tensor) product, AX + XB = W is equivalent to ( (Im A) + ( B T I n )) vec (X ) = vec (W ). Hence, Sylvester equation has a unique solution M := (I m A) + ( ) B T I n is invertible. 0 Λ (M) = Λ ((I m A) + (B T I n)) = {λ j + µ k, λ j Λ (A), µ k Λ (B)}. Λ (A) Λ ( B) = Corollary A, B Hurwitz = Sylvester equation has unique solution. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 6/39

15 Linear Matrix Equations Solvability Using the Kronecker (tensor) product, AX + XB = W is equivalent to ( (Im A) + ( B T I n )) vec (X ) = vec (W ). Hence, Sylvester equation has a unique solution M := (I m A) + ( ) B T I n is invertible. 0 Λ (M) = Λ ((I m A) + (B T I n)) = {λ j + µ k, λ j Λ (A), µ k Λ (B)}. Λ (A) Λ ( B) = Corollary A, B Hurwitz = Sylvester equation has unique solution. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 6/39

16 Linear Matrix Equations Solvability Using the Kronecker (tensor) product, AX + XB = W is equivalent to ( (Im A) + ( B T I n )) vec (X ) = vec (W ). Hence, Sylvester equation has a unique solution M := (I m A) + ( ) B T I n is invertible. 0 Λ (M) = Λ ((I m A) + (B T I n)) = {λ j + µ k, λ j Λ (A), µ k Λ (B)}. Λ (A) Λ ( B) = Corollary A, B Hurwitz = Sylvester equation has unique solution. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 6/39

17 Linear Matrix Equations Solvability Using the Kronecker (tensor) product, AX + XB = W is equivalent to ( (Im A) + ( B T I n )) vec (X ) = vec (W ). Hence, Sylvester equation has a unique solution M := (I m A) + ( ) B T I n is invertible. 0 Λ (M) = Λ ((I m A) + (B T I n)) = {λ j + µ k, λ j Λ (A), µ k Λ (B)}. Λ (A) Λ ( B) = Corollary A, B Hurwitz = Sylvester equation has unique solution. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 6/39

18 Linear Matrix Equations Complexity Issues Solving the Sylvester equation AX + XB = W via the equivalent linear system of equations ( (Im A) + ( B T I n )) vec (X ) = vec (W ) requires LU factorization of nm nm matrix; for n m, complexity is 2 3 n6 ; storing n m unknowns: for n m we have n 4 data for X. Example n = m = 1, 000 Gaussian elimination on an Intel core i7 (Westmere, 6 cores, 3.46 GHz 83.2 GFLOP peak) would take > 94 DAYS and 7.3 TB of memory! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 7/39

19 Linear Matrix Equations Complexity Issues Solving the Sylvester equation AX + XB = W via the equivalent linear system of equations ( (Im A) + ( B T I n )) vec (X ) = vec (W ) requires LU factorization of nm nm matrix; for n m, complexity is 2 3 n6 ; storing n m unknowns: for n m we have n 4 data for X. Example n = m = 1, 000 Gaussian elimination on an Intel core i7 (Westmere, 6 cores, 3.46 GHz 83.2 GFLOP peak) would take > 94 DAYS and 7.3 TB of memory! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 7/39

20 Numerical Methods for Solving Lyapunov Equations Traditional Methods Bartels-Stewart method for Sylvester and Lyapunov equation (lyap); Hessenberg-Schur method for Sylvester equations (lyap); Hammarling s method for Lyapunov equations AX + XA T + GG T = 0 with A Hurwitz (lyapchol). All based on the fact that if A, B T are in Schur form, then M = (I m A) + ( B T I n ) is block-upper triangular. Hence, solve Mx = b by back-substitution. Clever implementation of back-substitution process requires nm(n + m) flops. For Sylvester eqns., B in Hessenberg form is enough ( Hessenberg-Schur method). Hammarling s method computes Cholesky factor Y of X directly. All methods require Schur decomposition of A and Schur or Hessenberg decomposition of B need QR algorithm which requires 25n 3 flops for Schur decomposition. Not feasible for large-scale problems (n > 10, 000). Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 8/39

21 Numerical Methods for Solving Lyapunov Equations Traditional Methods Bartels-Stewart method for Sylvester and Lyapunov equation (lyap); Hessenberg-Schur method for Sylvester equations (lyap); Hammarling s method for Lyapunov equations AX + XA T + GG T = 0 with A Hurwitz (lyapchol). All based on the fact that if A, B T are in Schur form, then M = (I m A) + ( B T I n ) is block-upper triangular. Hence, solve Mx = b by back-substitution. Clever implementation of back-substitution process requires nm(n + m) flops. For Sylvester eqns., B in Hessenberg form is enough ( Hessenberg-Schur method). Hammarling s method computes Cholesky factor Y of X directly. All methods require Schur decomposition of A and Schur or Hessenberg decomposition of B need QR algorithm which requires 25n 3 flops for Schur decomposition. Not feasible for large-scale problems (n > 10, 000). Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 8/39

22 Numerical Methods for Solving Lyapunov Equations Traditional Methods Bartels-Stewart method for Sylvester and Lyapunov equation (lyap); Hessenberg-Schur method for Sylvester equations (lyap); Hammarling s method for Lyapunov equations AX + XA T + GG T = 0 with A Hurwitz (lyapchol). All based on the fact that if A, B T are in Schur form, then M = (I m A) + ( B T I n ) is block-upper triangular. Hence, solve Mx = b by back-substitution. Clever implementation of back-substitution process requires nm(n + m) flops. For Sylvester eqns., B in Hessenberg form is enough ( Hessenberg-Schur method). Hammarling s method computes Cholesky factor Y of X directly. All methods require Schur decomposition of A and Schur or Hessenberg decomposition of B need QR algorithm which requires 25n 3 flops for Schur decomposition. Not feasible for large-scale problems (n > 10, 000). Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 8/39

23 Numerical Methods for Solving Lyapunov Equations The Sign Function Method Definition For Z R n n with Λ (Z) ır = and Jordan canonical form [ ] J + 0 Z = S S 1 0 J the matrix sign function is [ Ik 0 sign (Z) := S 0 I n k ] S 1. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 9/39

24 Numerical Methods for Solving Lyapunov Equations The Sign Function Method Definition For Z R n n with Λ (Z) ır = and Jordan canonical form [ ] J + 0 Z = S S 1 0 J the matrix sign function is [ Ik 0 sign (Z) := S 0 I n k ] S 1. Lemma Let T R n n be nonsingular and Z as before, then sign ( TZT 1) = T sign (Z) T 1 Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 9/39

25 Numerical Methods for Solving Lyapunov Equations The Sign Function Method Computation of sign (Z) sign (Z) is root of I n = use Newton s method to compute it: Z 0 Z, Z j+1 1 ( c j Z j + 1 ) Z 1 j, j = 1, 2,... 2 c j = sign (Z) = lim j Z j. c j > 0 is scaling parameter for convergence acceleration and rounding error minimization, e.g. Z 1 j F c j =, Z j F based on equilibrating the norms of the two summands [Higham 86]. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 9/39

26 Solving Lyapunov Equations with the Matrix Sign Function Method Key observation: If X R n n is a solution of AX + XA T + W = 0, then [ ] [ ] [ ] [ In X A W In X A 0 = 0 I n 0 A T 0 I n 0 A T }{{}}{{}}{{} =T 1 =:H =:T ]. Hence, if A is Hurwitz (i.e., asymptotically stable), then ( [ ] ) ([ A 0 A 0 sign (H) = sign T T 1 = T sign 0 A T 0 A T [ ] In 2X =. 0 I n ]) T 1 Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 10/39

27 Solving Lyapunov Equations with the Matrix Sign Function Method Key observation: If X R n n is a solution of AX + XA T + W = 0, then [ ] [ ] [ ] [ In X A W In X A 0 = 0 I n 0 A T 0 I n 0 A T }{{}}{{}}{{} =T 1 =:H =:T ]. Hence, if A is Hurwitz (i.e., asymptotically stable), then ( [ ] ) ([ A 0 A 0 sign (H) = sign T T 1 = T sign 0 A T 0 A T [ ] In 2X =. 0 I n ]) T 1 Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 10/39

28 Solving Lyapunov Equations with the Matrix Sign Function Method [ ] A W Apply sign function iteration Z 1 2 (Z + Z 1 ) to H = : 0 A T H + H 1 = [ A W 0 A T ] + [ A 1 A 1 WA T 0 A T ] = Sign function iteration for Lyapunov equation: ( ) A 0 A, A j A j + A 1 j, ( ) W 0 G, W j W j + A 1 j W j A T j, j = 0, 1, 2,.... Define A := lim j A j, W := lim j W j. Theorem If A is Hurwitz, then A = I n and X = 1 2 W. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 11/39

29 Solving Lyapunov Equations with the Matrix Sign Function Method Factored form Recall sign function iteration for AX + XA T + W = 0: ( ) A 0 A, A j+1 1 Aj + A 1 2 j, ( ) j = 0, 1, 2,.... W 0 G, W j+1 1 Wj + A 1 2 j W j A T j, Now consider the second iteration for W = BB T, starting with W 0 = BB T =: B 0B T 0 : 1 ( ) W j + A 1 j W j A T j 2 Hence, obtain factored iteration = 1 2 = 1 2 with S := 1 2 lim j B j and X = SS T. ( B j B T j B j [ Bj A 1 j B j ] ) + A 1 j B j Bj T A T j [ Bj A 1 j B j ] [ Bj A 1 j B j ] T. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 12/39

30 Solving Lyapunov Equations with the Matrix Sign Function Method Factored form Recall sign function iteration for AX + XA T + W = 0: ( ) A 0 A, A j+1 1 Aj + A 1 2 j, ( ) j = 0, 1, 2,.... W 0 G, W j+1 1 Wj + A 1 2 j W j A T j, Now consider the second iteration for W = BB T, starting with W 0 = BB T =: B 0B T 0 : 1 ( ) W j + A 1 j W j A T j 2 Hence, obtain factored iteration = 1 2 = 1 2 with S := 1 2 lim j B j and X = SS T. ( B j B T j B j [ Bj A 1 j B j ] ) + A 1 j B j Bj T A T j [ Bj A 1 j B j ] [ Bj A 1 j B j ] T. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 12/39

31 Solving Lyapunov Equations with the Matrix Sign Function Method Factored form Recall sign function iteration for AX + XA T + W = 0: ( ) A 0 A, A j+1 1 Aj + A 1 2 j, ( ) j = 0, 1, 2,.... W 0 G, W j+1 1 Wj + A 1 2 j W j A T j, Now consider the second iteration for W = BB T, starting with W 0 = BB T =: B 0B T 0 : 1 ( ) W j + A 1 j W j A T j 2 Hence, obtain factored iteration = 1 2 = 1 2 with S := 1 2 lim j B j and X = SS T. ( B j B T j B j [ Bj A 1 j B j ] ) + A 1 j B j Bj T A T j [ Bj A 1 j B j ] [ Bj A 1 j B j ] T. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 12/39

32 Solving Lyapunov Equations with the Matrix Sign Function Method Factored form Recall sign function iteration for AX + XA T + W = 0: ( ) A 0 A, A j+1 1 Aj + A 1 2 j, ( ) j = 0, 1, 2,.... W 0 G, W j+1 1 Wj + A 1 2 j W j A T j, Now consider the second iteration for W = BB T, starting with W 0 = BB T =: B 0B T 0 : 1 ( ) W j + A 1 j W j A T j 2 Hence, obtain factored iteration = 1 2 = 1 2 with S := 1 2 lim j B j and X = SS T. ( B j B T j B j [ Bj A 1 j B j ] ) + A 1 j B j Bj T A T j [ Bj A 1 j B j ] [ Bj A 1 j B j ] T. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 12/39

33 Solving Lyapunov Equations with the Matrix Sign Function Method Factored form [B./Quintana-Ortí 97] Factored sign function iteration for A(SS T ) + (SS T )A T + BB T = 0 ( ) A 0 A, A j A j + A 1 j, [ B 0 B, B j+1 1 Bj 2 A 1 ] j = 0, 1, 2,.... j B j, Remarks: To get both Gramians, run in parallel C j+1 1 [ ] Cj 2 C j A 1. j To avoid growth in numbers of columns of B j (or rows of C j ): column compression by RRLQ or truncated SVD. Several options to incorporate scaling, e.g., scale A -iteration only. Simple stopping cirterion: A j + I n F tol. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 13/39

34 Numerical Methods for Solving Lyapunov Equations The ADI Method Recall Peaceman Rachford ADI: Consider Au = s where A R n n spd, s R n. ADI Iteration Idea: Decompose A = H + V with H, V R n n such that can be solved easily/efficiently. ADI Iteration (H + pi )v = r (V + pi )w = t If H, V spd p k, k = 1, 2,... such that u 0 = 0 (H + p k I )u k 1 = (p 2 k I V )u k 1 + s (V + p k I )u k = (p k I H)u k 1 + s 2 converges to u R n solving Au = s. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 14/39

35 Numerical Methods for Solving Lyapunov Equations The ADI Method Recall Peaceman Rachford ADI: Consider Au = s where A R n n spd, s R n. ADI Iteration Idea: Decompose A = H + V with H, V R n n such that can be solved easily/efficiently. ADI Iteration (H + pi )v = r (V + pi )w = t If H, V spd p k, k = 1, 2,... such that u 0 = 0 (H + p k I )u k 1 = (p 2 k I V )u k 1 + s (V + p k I )u k = (p k I H)u k 1 + s 2 converges to u R n solving Au = s. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 14/39

36 Numerical Methods for Solving Lyapunov Equations The Lyapunov operator L : P AX + XA T can be decomposed into the linear operators L H : X AX, L V : X XA T. In analogy to the standard ADI method we find the ADI iteration for the Lyapunov equation [Wachspress 88] X 0 = 0 (A + p k I )X k 1 = W X 2 k 1 (A T p k I ) (A + p k I )Xk T = W X T (A T p k 1 k I ). 2 Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 15/39

37 Numerical Methods for Solving Lyapunov Equations Low-Rank ADI Consider AX + XA T = BB T for stable A; B R n m with m n. ADI iteration for the Lyapunov equation [Wachspress 95] For k = 1,..., k max X 0 = 0 (A + p k I )X k 1 = BB T X 2 k 1 (A T p k I ) (A + p k I )Xk T = BB T X T (A T p k 1 k I ) 2 Rewrite as one step iteration and factorize X k = Z k Z T k, k = 0,..., k max Z 0 Z0 T = 0 Z k Zk T = 2p k (A + p k I ) 1 BB T (A + p k I ) T +(A + p k I ) 1 (A p k I )Z k 1 Zk 1 T (A p ki ) T (A + p k I ) T... low-rank Cholesky factor ADI [Penzl 97/ 00, Li/White 99/ 02, B./Li/Penzl 99/ 08, Gugercin/Sorensen/Antoulas 03] Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 16/39

38 Numerical Methods for Solving Lyapunov Equations Low-Rank ADI Consider AX + XA T = BB T for stable A; B R n m with m n. ADI iteration for the Lyapunov equation [Wachspress 95] For k = 1,..., k max X 0 = 0 (A + p k I )X k 1 = BB T X 2 k 1 (A T p k I ) (A + p k I )Xk T = BB T X T (A T p k 1 k I ) 2 Rewrite as one step iteration and factorize X k = Z k Z T k, k = 0,..., k max Z 0 Z0 T = 0 Z k Zk T = 2p k (A + p k I ) 1 BB T (A + p k I ) T +(A + p k I ) 1 (A p k I )Z k 1 Zk 1 T (A p ki ) T (A + p k I ) T... low-rank Cholesky factor ADI [Penzl 97/ 00, Li/White 99/ 02, B./Li/Penzl 99/ 08, Gugercin/Sorensen/Antoulas 03] Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 16/39

39 Numerical Methods for Solving Lyapunov Equations Low-Rank ADI Consider AX + XA T = BB T for stable A; B R n m with m n. ADI iteration for the Lyapunov equation [Wachspress 95] For k = 1,..., k max X 0 = 0 (A + p k I )X k 1 = BB T X 2 k 1 (A T p k I ) (A + p k I )Xk T = BB T X T (A T p k 1 k I ) 2 Rewrite as one step iteration and factorize X k = Z k Z T k, k = 0,..., k max Z 0 Z0 T = 0 Z k Zk T = 2p k (A + p k I ) 1 BB T (A + p k I ) T +(A + p k I ) 1 (A p k I )Z k 1 Zk 1 T (A p ki ) T (A + p k I ) T... low-rank Cholesky factor ADI [Penzl 97/ 00, Li/White 99/ 02, B./Li/Penzl 99/ 08, Gugercin/Sorensen/Antoulas 03] Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 16/39

40 Solving Large-Scale Matrix Equations Numerical Methods for Solving Lyapunov Equations Z k = [ 2p k (A + p k I ) 1 B, (A + p k I ) 1 (A p k I )Z k 1 ] [Penzl 00] Observing that (A p i I ), (A + p k I ) 1 commute, we rewrite Z kmax as Z kmax = [z kmax, P kmax 1z kmax, P kmax 2(P kmax 1z kmax ),..., P 1(P 2 P kmax 1z kmax )], where and P i := z kmax = 2p kmax (A + p kmax I ) 1 B 2pi 2pi+1 [ I (pi + p i+1 )(A + p i I ) 1]. [Li/White 02] Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 17/39

41 Solving Large-Scale Matrix Equations Numerical Methods for Solving Lyapunov Equations Z k = [ 2p k (A + p k I ) 1 B, (A + p k I ) 1 (A p k I )Z k 1 ] [Penzl 00] Observing that (A p i I ), (A + p k I ) 1 commute, we rewrite Z kmax as Z kmax = [z kmax, P kmax 1z kmax, P kmax 2(P kmax 1z kmax ),..., P 1(P 2 P kmax 1z kmax )], where and P i := z kmax = 2p kmax (A + p kmax I ) 1 B 2pi 2pi+1 [ I (pi + p i+1 )(A + p i I ) 1]. [Li/White 02] Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 17/39

42 Numerical Methods for Solving Lyapunov Equations Lyapunov equation 0 = AX + XA T + BB T. Algorithm [Penzl 97/ 00, Li/White 99/ 02, B. 04, B./Li/Penzl 99/ 08] V 1 2 re p 1(A + p 1I ) 1 B, Z 1 V 1 FOR k = 2, 3,... V k re p k re p k 1 ( Vk 1 (p k + p k 1 )(A + p k I ) 1 V k 1 ) Z k [ Z k 1 V k ] Z k rrlq(z k, τ) column compression At convergence, Z kmax Zk T max X, where (without column compression) Z kmax = [ V 1... V kmax ], Vk = C n m. Note: Implementation in real arithmetic possible by combining two steps [B./Li/Penzl 99/ 08] or using new idea employing the relation of 2 consecutive complex factors [B./Kürschner/Saak 11]. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 18/39

43 Numerical Methods for Solving Lyapunov Equations Lyapunov equation 0 = AX + XA T + BB T. Algorithm [Penzl 97/ 00, Li/White 99/ 02, B. 04, B./Li/Penzl 99/ 08] V 1 2 re p 1(A + p 1I ) 1 B, Z 1 V 1 FOR k = 2, 3,... V k re p k re p k 1 ( Vk 1 (p k + p k 1 )(A + p k I ) 1 V k 1 ) Z k [ Z k 1 V k ] Z k rrlq(z k, τ) column compression At convergence, Z kmax Zk T max X, where (without column compression) Z kmax = [ V 1... V kmax ], Vk = C n m. Note: Implementation in real arithmetic possible by combining two steps [B./Li/Penzl 99/ 08] or using new idea employing the relation of 2 consecutive complex factors [B./Kürschner/Saak 11]. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 18/39

44 Numerical Results for ADI Optimal Cooling of Steel Profiles Mathematical model: boundary control for linearized 2D heat equation. c ρ t x = λ x, ξ Ω λ n x = κ(u k x), ξ Γ k, 1 k 7, x = 0, ξ Γ7. n = m = 7, q = FEM Discretization, different models for initial mesh (n = 371), 1, 2, 3, 4 steps of mesh refinement n = 1357, 5177, 20209, Source: Physical model: courtesy of Mannesmann/Demag. Math. model: Tröltzsch/Unger 1999/2001, Penzl 1999, Saak Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 19/39

45 Numerical Results for ADI Optimal Cooling of Steel Profiles Solve dual Lyapunov equations needed for balanced truncation, i.e., APM T + MPA T + BB T = 0, A T QM + M T QA + C T C = 0, for n = 79, shifts chosen by Penzl heuristic from 50/25 Ritz values of A of largest/smallest magnitude, no column compression performed. No factorization of mass matrix required. Computations done on Core2Duo at 2.8GHz with 3GB RAM and 32Bit-MATLAB. CPU times: 626 / 356 sec. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 20/39

46 Numerical Results for ADI Scaling / Mesh Independence Computations by Martin Köhler 10 A R n n FDM matrix for 2D heat equation on [0, 1] 2 (Lyapack benchmark demo l1, m = 1). 16 shifts chosen by Penzl heuristic from 50/25 Ritz values of A of largest/smallest magnitude. Computations on 2 dual core Intel Xeon 5160 with 16 GB RAM using M.E.S.S. ( Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 21/39

47 Numerical Results for ADI Scaling / Mesh Independence Computations by Martin Köhler 10 A R n n FDM matrix for 2D heat equation on [0, 1] 2 (Lyapack benchmark demo l1, m = 1). 16 shifts chosen by Penzl heuristic from 50/25 Ritz values of A of largest/smallest magnitude. Computations on 2 dual core Intel Xeon 5160 with 16 GB RAM using M.E.S.S. ( CPU Times n M.E.S.S. 1 (C) LyaPack M.E.S.S. (MATLAB) , , , , , out of memory , out of memory , out of memory ,000, out of memory Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 21/39

48 Numerical Results for ADI Scaling / Mesh Independence Computations by Martin Köhler 10 A R n n FDM matrix for 2D heat equation on [0, 1] 2 (Lyapack benchmark demo l1, m = 1). 16 shifts chosen by Penzl heuristic from 50/25 Ritz values of A of largest/smallest magnitude. Computations on 2 dual core Intel Xeon 5160 with 16 GB RAM using M.E.S.S. ( Note: for n = 1, 000, 000, first sparse LU needs 1, 100 sec., using UMFPACK this reduces to 30 sec. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 21/39

49 Factored Galerkin-ADI Iteration Lyapunov equation 0 = AX + XA T + BB T Projection-based methods for Lyapunov equations with A + A T < 0: 1 Compute orthonormal basis range (Z), Z R n r, for subspace Z R n, dim Z = r. 2 Set  := Z T AZ, ˆB := Z T B. 3 Solve small-size Lyapunov equation  ˆX + ˆX ÂT + ˆB ˆB T = 0. 4 Use X Z ˆX Z T. Examples: Krylov subspace methods, i.e., for m = 1: Z = K(A, B, r) = span{b, AB, A 2 B,..., A r 1 B} [Saad 90, Jaimoukha/Kasenally 94, Jbilou 02 08]. K-PIK [Simoncini 07], Z = K(A, B, r) K(A 1, B, r). Rational Krylov [Druskin/Simoncini 11] ( exercises). Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 22/39

50 Factored Galerkin-ADI Iteration Lyapunov equation 0 = AX + XA T + BB T Projection-based methods for Lyapunov equations with A + A T < 0: 1 Compute orthonormal basis range (Z), Z R n r, for subspace Z R n, dim Z = r. 2 Set  := Z T AZ, ˆB := Z T B. 3 Solve small-size Lyapunov equation  ˆX + ˆX ÂT + ˆB ˆB T = 0. 4 Use X Z ˆX Z T. Examples: Krylov subspace methods, i.e., for m = 1: Z = K(A, B, r) = span{b, AB, A 2 B,..., A r 1 B} [Saad 90, Jaimoukha/Kasenally 94, Jbilou 02 08]. K-PIK [Simoncini 07], Z = K(A, B, r) K(A 1, B, r). Rational Krylov [Druskin/Simoncini 11] ( exercises). Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 22/39

51 Factored Galerkin-ADI Iteration Lyapunov equation 0 = AX + XA T + BB T Projection-based methods for Lyapunov equations with A + A T < 0: 1 Compute orthonormal basis range (Z), Z R n r, for subspace Z R n, dim Z = r. 2 Set  := Z T AZ, ˆB := Z T B. 3 Solve small-size Lyapunov equation  ˆX + ˆX ÂT + ˆB ˆB T = 0. 4 Use X Z ˆX Z T. Examples: Krylov subspace methods, i.e., for m = 1: Z = K(A, B, r) = span{b, AB, A 2 B,..., A r 1 B} [Saad 90, Jaimoukha/Kasenally 94, Jbilou 02 08]. K-PIK [Simoncini 07], Z = K(A, B, r) K(A 1, B, r). Rational Krylov [Druskin/Simoncini 11] ( exercises). Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 22/39

52 Factored Galerkin-ADI Iteration Lyapunov equation 0 = AX + XA T + BB T Projection-based methods for Lyapunov equations with A + A T < 0: 1 Compute orthonormal basis range (Z), Z R n r, for subspace Z R n, dim Z = r. 2 Set  := Z T AZ, ˆB := Z T B. 3 Solve small-size Lyapunov equation  ˆX + ˆX ÂT + ˆB ˆB T = 0. 4 Use X Z ˆX Z T. Examples: ADI subspace [B./R.-C. Li/Truhar 08]: Note: Z = colspan [ V 1,..., V r ]. 1 ADI subspace is rational Krylov subspace [J.-R. Li/White 02]. 2 Similar approach: ADI-preconditioned global Arnoldi method [Jbilou 08]. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 22/39

53 Numerical Methods for Solving Lyapunov Equations Numerical examples for Galerkin-ADI FEM semi-discretized control problem for parabolic PDE: optimal cooling of rail profiles, n = 20, 209, m = 7, q = 6. Good ADI shifts CPU times: 80s (projection every 5th ADI step) vs. 94s (no projection). Computations by Jens Saak 10. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 23/39

54 Numerical Methods for Solving Lyapunov Equations Numerical examples for Galerkin-ADI FEM semi-discretized control problem for parabolic PDE: optimal cooling of rail profiles, n = 20, 209, m = 7, q = 6. Bad ADI shifts CPU times: 368s (projection every 5th ADI step) vs. 1207s (no projection). Computations by Jens Saak 10. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 23/39

55 Numerical Methods for Solving Lyapunov Equations Numerical examples for Galerkin-ADI: optimal cooling of rail profiles, n = 79, 841. M.E.S.S. w/o Galerkin projection and column compression Rank of solution factors: 532 / 426 M.E.S.S. with Galerkin projection and column compression Rank of solution factors: 269 / 205 Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 24/39

56 Solving Large-Scale Matrix Equations Numerical example for BT: Optimal Cooling of Steel Profiles n = 1, 357, Absolute Error BT model computed with sign function method, MT w/o static condensation, same order as BT model. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 25/39

57 Solving Large-Scale Matrix Equations Numerical example for BT: Optimal Cooling of Steel Profiles n = 1, 357, Absolute Error n = 79, 841, Absolute Error BT model computed with sign function method, MT w/o static condensation, same order as BT model. BT model computed using M.E.S.S. in MATLAB, dualcore, computation time: <10 min. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 25/39

58 Solving Large-Scale Matrix Equations Numerical example for BT: Microgyroscope (Butterfly Gyro) FEM discretization of structure dynamical model using quadratic tetrahedral elements (ANSYS-SOLID187) n = 34, 722, m = 1, q = 12. Reduced model computed using SpaRed, r = 30. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 26/39

59 Solving Large-Scale Matrix Equations Numerical example for BT: Microgyroscope (Butterfly Gyro) FEM discretization of structure dynamical model using quadratic tetrahedral elements (ANSYS-SOLID187) n = 34, 722, m = 1, q = 12. Reduced model computed using SpaRed, r = 30. Frequency Repsonse Analysis Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 26/39

60 Solving Large-Scale Matrix Equations Numerical example for BT: Microgyroscope (Butterfly Gyro) FEM discretization of structure dynamical model using quadratic tetrahedral elements (ANSYS-SOLID187) n = 34, 722, m = 1, q = 12. Reduced model computed using SpaRed, r = 30. Frequency Repsonse Analysis Hankel Singular Values Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 26/39

61 Solving Large-Scale Algebraic Riccati Equations Theory [Lancaster/Rodman 95] Theorem Consider the (continuous-time) algebraic Riccati equation (ARE) 0 = R(X ) = C T C + A T X + XA XBB T X, with A R n n, B R n m, C R q n, (A, B) stabilizable, (A, C) detectable. Then: (a) There exists a unique stabilizing X {X R n n R(X ) = 0 }, i.e., Λ (A BB T X ) C. (b) X = X T 0 and X X for all X {X R n n R(X ) = 0 }. (c) If (A, C) observable, then X > 0. {[ ]} I n (d) span is the unique invariant subspace of the Hamiltonian X matrix [ ] A BB T H = C T C A T corresponding to Λ (H) C. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 27/39

62 Solving Large-Scale Algebraic Riccati Equations Numerical Methods [Bini/Iannazzo/Meini 12] Numerical Methods (incomplete list) Invariant subspace methods ( eigenproblem for Hamiltonian matrix): Schur vector method (care) [Laub 79] Hamiltonian SR algorithm [Bunse-Gerstner/Mehrmann 86] Symplectic URV-based method [B./Mehrmann/Xu 97/ 98, Chu/Liu/Mehrmann 07] Spectral projection methods Sign function method [Roberts 71, Byers 87] Disk function method [Bai/Demmel/Gu 94, B. 97] (rational, global) Krylov subspace techniques [Jaimoukha/Kasenally 94, Jbilou 03/ 06, Heyouni/Jbilou 09] Newton s method Kleinman iteration [Kleinman 68] Line search acceleration [B./Byers 98] Newton-ADI [B./J.-R. Li/Penzl 99/ 08] Inexact Newton [Feitzinger/Hylla/Sachs 09] Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 28/39

63 Solving Large-Scale Algebraic Riccati Equations Newton s Method for AREs [Kleinman 68, Mehrmann 91, Lancaster/Rodman 95, B./Byers 94/ 98, B. 97, Guo/Laub 99] Consider 0 = R(X ) = C T C + A T X + XA XBB T X. Frechét derivative of R(X ) at X : R X : Z (A BBT X ) T Z + Z(A BB T X ). Newton-Kantorovich method: ( ) 1 X j+1 = X j R X R(Xj j ), j = 0, 1, 2,... Newton s method (with line search) for AREs FOR j = 0, 1,... 1 A j A BB T X j =: A BK j. 2 Solve the Lyapunov equation A T j N j + N j A j = R(X j ). 3 X j+1 X j + t j N j. END FOR j Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 29/39

64 Solving Large-Scale Algebraic Riccati Equations Newton s Method for AREs [Kleinman 68, Mehrmann 91, Lancaster/Rodman 95, B./Byers 94/ 98, B. 97, Guo/Laub 99] Consider 0 = R(X ) = C T C + A T X + XA XBB T X. Frechét derivative of R(X ) at X : R X : Z (A BBT X ) T Z + Z(A BB T X ). Newton-Kantorovich method: ( ) 1 X j+1 = X j R X R(Xj j ), j = 0, 1, 2,... Newton s method (with line search) for AREs FOR j = 0, 1,... 1 A j A BB T X j =: A BK j. 2 Solve the Lyapunov equation A T j N j + N j A j = R(X j ). 3 X j+1 X j + t j N j. END FOR j Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 29/39

65 Solving Large-Scale Algebraic Riccati Equations Newton s Method for AREs [Kleinman 68, Mehrmann 91, Lancaster/Rodman 95, B./Byers 94/ 98, B. 97, Guo/Laub 99] Consider 0 = R(X ) = C T C + A T X + XA XBB T X. Frechét derivative of R(X ) at X : R X : Z (A BBT X ) T Z + Z(A BB T X ). Newton-Kantorovich method: ( ) 1 X j+1 = X j R X R(Xj j ), j = 0, 1, 2,... Newton s method (with line search) for AREs FOR j = 0, 1,... 1 A j A BB T X j =: A BK j. 2 Solve the Lyapunov equation A T j N j + N j A j = R(X j ). 3 X j+1 X j + t j N j. END FOR j Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 29/39

66 Solving Large-Scale Algebraic Riccati Equations Newton s Method for AREs [Kleinman 68, Mehrmann 91, Lancaster/Rodman 95, B./Byers 94/ 98, B. 97, Guo/Laub 99] Consider 0 = R(X ) = C T C + A T X + XA XBB T X. Frechét derivative of R(X ) at X : R X : Z (A BBT X ) T Z + Z(A BB T X ). Newton-Kantorovich method: ( ) 1 X j+1 = X j R X R(Xj j ), j = 0, 1, 2,... Newton s method (with line search) for AREs FOR j = 0, 1,... 1 A j A BB T X j =: A BK j. 2 Solve the Lyapunov equation A T j N j + N j A j = R(X j ). 3 X j+1 X j + t j N j. END FOR j Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 29/39

67 Newton s Method for AREs Properties and Implementation Convergence for K 0 stabilizing: A j = A BK j = A BB T X j is stable j 0. lim j R(X j ) F = 0 (monotonically). lim j X j = X 0 (locally quadratic). Need large-scale Lyapunov solver; here, ADI iteration: linear systems with dense, but sparse+low rank coefficient matrix A j : A j = A B K j = sparse m m n = efficient inversion using Sherman-Morrison-Woodbury formula: (A BK j + p (j) k I ) 1 = (I n + (A + p (j) k I ) 1 B(I m K j (A + p (j) k I ) 1 B) 1 K j )(A + p (j) k I ) 1. BUT: X = X T R n n = n(n + 1)/2 unknowns! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 30/39

68 Newton s Method for AREs Properties and Implementation Convergence for K 0 stabilizing: A j = A BK j = A BB T X j is stable j 0. lim j R(X j ) F = 0 (monotonically). lim j X j = X 0 (locally quadratic). Need large-scale Lyapunov solver; here, ADI iteration: linear systems with dense, but sparse+low rank coefficient matrix A j : A j = A B K j = sparse m m n = efficient inversion using Sherman-Morrison-Woodbury formula: (A BK j + p (j) k I ) 1 = (I n + (A + p (j) k I ) 1 B(I m K j (A + p (j) k I ) 1 B) 1 K j )(A + p (j) k I ) 1. BUT: X = X T R n n = n(n + 1)/2 unknowns! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 30/39

69 Newton s Method for AREs Properties and Implementation Convergence for K 0 stabilizing: A j = A BK j = A BB T X j is stable j 0. lim j R(X j ) F = 0 (monotonically). lim j X j = X 0 (locally quadratic). Need large-scale Lyapunov solver; here, ADI iteration: linear systems with dense, but sparse+low rank coefficient matrix A j : A j = A B K j = sparse m m n = efficient inversion using Sherman-Morrison-Woodbury formula: (A BK j + p (j) k I ) 1 = (I n + (A + p (j) k I ) 1 B(I m K j (A + p (j) k I ) 1 B) 1 K j )(A + p (j) k I ) 1. BUT: X = X T R n n = n(n + 1)/2 unknowns! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 30/39

70 Newton s Method for AREs Properties and Implementation Convergence for K 0 stabilizing: A j = A BK j = A BB T X j is stable j 0. lim j R(X j ) F = 0 (monotonically). lim j X j = X 0 (locally quadratic). Need large-scale Lyapunov solver; here, ADI iteration: linear systems with dense, but sparse+low rank coefficient matrix A j : A j = A B K j = sparse m m n = efficient inversion using Sherman-Morrison-Woodbury formula: (A BK j + p (j) k I ) 1 = (I n + (A + p (j) k I ) 1 B(I m K j (A + p (j) k I ) 1 B) 1 K j )(A + p (j) k I ) 1. BUT: X = X T R n n = n(n + 1)/2 unknowns! Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 30/39

71 Low-Rank Newton-ADI for AREs Re-write Newton s method for AREs A T j (X j + N j ) }{{} A T j N j + N j A j = R(X j ) + (X j + N j ) }{{} =X j+1 A j = C T C X j BB T X j }{{} =X j+1 =: W j W T j Set X j = Z j Zj T for rank (Z j ) n = A T j ( Zj+1 Zj+1 T ) ( + Zj+1 Zj+1 T ) Aj = W j Wj T Factored Newton Iteration [B./Li/Penzl 1999/2008] Solve Lyapunov equations for Z j+1 directly by factored ADI iteration and use sparse + low-rank structure of A j. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 31/39

72 Low-Rank Newton-ADI for AREs Re-write Newton s method for AREs A T j (X j + N j ) }{{} A T j N j + N j A j = R(X j ) + (X j + N j ) }{{} =X j+1 A j = C T C X j BB T X j }{{} =X j+1 =: W j W T j Set X j = Z j Zj T for rank (Z j ) n = A T j ( Zj+1 Zj+1 T ) ( + Zj+1 Zj+1 T ) Aj = W j Wj T Factored Newton Iteration [B./Li/Penzl 1999/2008] Solve Lyapunov equations for Z j+1 directly by factored ADI iteration and use sparse + low-rank structure of A j. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 31/39

73 Low-Rank Newton-ADI for AREs Feedback Iteration Optimal feedback K = B T X = B T Z Z T can be computed by direct feedback iteration: jth Newton iteration: k max K j = B T Z j Zj T = (B T V j,k )Vj,k T k=1 j K = B T Z Z T K j can be updated in ADI iteration, no need to even form Z j, need only fixed workspace for K j R m n! Related to earlier work by [Banks/Ito 1991]. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 32/39

74 Solving Large-Scale Matrix Equations Galerkin-Newton-ADI Basic ideas Hybrid method of Galerkin projection methods for AREs [Jaimoukha/Kasenally 94, Jbilou 06, Heyouni/Jbilou 09] and Newton-ADI, i.e., use column space of current Newton iterate for projection, solve projected ARE, and prolongate. Independence of good parameters observed for Galerkin-ADI applied to Lyapunov equations fix ADI parameters for all Newton iterations. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 33/39

75 Solving Large-Scale Matrix Equations Galerkin-Newton-ADI Basic ideas Hybrid method of Galerkin projection methods for AREs [Jaimoukha/Kasenally 94, Jbilou 06, Heyouni/Jbilou 09] and Newton-ADI, i.e., use column space of current Newton iterate for projection, solve projected ARE, and prolongate. Independence of good parameters observed for Galerkin-ADI applied to Lyapunov equations fix ADI parameters for all Newton iterations. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 33/39

76 Numerical Results LQR Problem for 2D Geometry Linear 2D heat equation with homogeneous Dirichlet boundary and point control/observation. FD discretization on uniform grid. n = , m = p = 1, 10 shifts for ADI iterations. Convergence of large-scale matrix equation solvers: Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 34/39

77 Numerical Results Newton-ADI vs. Newton-ADI-Gelerkin FDM for 2D heat/convection-diffusion equations on [0, 1] 2 (Lyapack benchmarks, m = p = 1) symmetric/nonsymmetric A R n n, n = 10, shifts chosen by Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A. Computations using Intel Core 2 Quad CPU of type Q9400 at 2.66GHz with 4 GB RAM and 64Bit-MATLAB. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 35/39

78 Numerical Results Newton-ADI vs. Newton-ADI-Gelerkin FDM for 2D heat/convection-diffusion equations on [0, 1] 2 (Lyapack benchmarks, m = p = 1) symmetric/nonsymmetric A R n n, n = 10, shifts chosen by Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A. Computations using Intel Core 2 Quad CPU of type Q9400 at 2.66GHz with 4 GB RAM and 64Bit-MATLAB. Newton-ADI step rel. change rel. residual ADI e e e e e e e e e e e e e e e e e e e CPU time: 76.9 sec. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 35/39

79 Numerical Results Newton-ADI vs. Newton-ADI-Gelerkin FDM for 2D heat/convection-diffusion equations on [0, 1] 2 (Lyapack benchmarks, m = p = 1) symmetric/nonsymmetric A R n n, n = 10, shifts chosen by Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A. Computations using Intel Core 2 Quad CPU of type Q9400 at 2.66GHz with 4 GB RAM and 64Bit-MATLAB. Newton-ADI step rel. change rel. residual ADI e e e e e e e e e e e e e e e e e e e CPU time: 76.9 sec. Newton-Galerkin-ADI step rel. change rel. residual ADI e e e e e e e e e e e e e e e CPU time: 38.0 sec. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 35/39

80 Numerical Results Newton-ADI vs. Newton-ADI-Gelerkin FDM for 2D heat/convection-diffusion equations on [0, 1] 2 (Lyapack benchmarks, m = p = 1) symmetric/nonsymmetric A R n n, n = 10, shifts chosen by Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A. Computations using Intel Core 2 Quad CPU of type Q9400 at 2.66GHz with 4 GB RAM and 64Bit-MATLAB relative change in LRCF no projection ever step every 5th step iteration index relative residual norm no projection ever step every 5th step iteration index Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 35/39

81 Numerical Results Newton-ADI vs. Newton-ADI-Gelerkin FDM for 2D heat/convection-diffusion equations on [0, 1] 2 (Lyapack benchmarks, m = p = 1) symmetric/nonsymmetric A R n n, n = 10, shifts chosen by Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A. Computations using Intel Core 2 Quad CPU of type Q9400 at 2.66GHz with 4 GB RAM and 64Bit-MATLAB. Newton-ADI step rel. change rel. residual ADI e e e e e e e e e e e e e e e CPU time: sec. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 35/39

82 Numerical Results Newton-ADI vs. Newton-ADI-Gelerkin FDM for 2D heat/convection-diffusion equations on [0, 1] 2 (Lyapack benchmarks, m = p = 1) symmetric/nonsymmetric A R n n, n = 10, shifts chosen by Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A. Computations using Intel Core 2 Quad CPU of type Q9400 at 2.66GHz with 4 GB RAM and 64Bit-MATLAB. Newton-ADI step rel. change rel. residual ADI e e e e e e e e e e e e e e e CPU time: sec. Newton-Galerkin-ADI step rel. change rel. residual ADI it e e e e e e e e e e e e e CPU time: 75.7 sec. Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 35/39

83 Numerical Results Newton-ADI vs. Newton-ADI-Gelerkin FDM for 2D heat/convection-diffusion equations on [0, 1] 2 (Lyapack benchmarks, m = p = 1) symmetric/nonsymmetric A R n n, n = 10, shifts chosen by Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A. Computations using Intel Core 2 Quad CPU of type Q9400 at 2.66GHz with 4 GB RAM and 64Bit-MATLAB relative change in LRCF no projection ever step every 5th step iteration index relative residual norm no projection ever step every 5th step iteration index Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 35/39

84 Numerical Results Example: LQR Problem for 3D Geometry Control problem for 3d Convection-Diffusion Equation FDM for 3D convection-diffusion equation on [0, 1] 3 proposed in [Simoncini 07], q = p = 1 non-symmetric A R n n, n = Test system: INTEL Xeon GHz ; 16 GB RAM; 64Bit-MATLAB (R2010a) using threaded BLAS; stopping tolerance: Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 36/39

85 Numerical Results Example: LQR Problem for 3D Geometry Newton-ADI NWT rel. change rel. residual ADI CPU time: sec. NG-ADI inner= 5, outer= 1 NWT rel. change rel. residual ADI CPU time: sec. NG-ADI inner= 1, outer= 1 NWT rel. change rel. residual ADI CPU time: sec. NG-ADI inner= 0, outer= 1 NWT rel. change rel. residual ADI CPU time: sec. Test system: INTEL Xeon GHz ; 16 GB RAM; 64Bit-MATLAB (R2010a) using threaded BLAS; stopping tolerance: Max Planck Institute Magdeburg Peter Benner, Lihong Feng, MOR for Dynamical Systems 36/39