A Newton-Galerkin-ADI Method for Large-Scale Algebraic Riccati Equations

Transcription

1 A Newton-Galerkin-ADI Method for Large-Scale Algebraic Riccati Equations Peter Benner Max-Planck-Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory Group Magdeburg, Germany Jens Saak Technische Universität Chemnitz Fakultät für Mathematik Mathematik in Industrie und Technik Chemnitz, Germany Applied Linear Algebra 2010 GAMM Workshop Applied and Numerical Linear Algebra Novi Sad, May 27, /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

2 Outline 1 Introduction 2 LRCF-ADI with Galerkin-Projection-Acceleration 3 LRCF-NM for the ARE 2/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

3 Introduction Large-Scale Algebraic Lyapunov and Riccati Equations General form of algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: 0 = R(X ) := A T X + XA XGX + W. 3/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

4 Introduction Large-Scale Algebraic Lyapunov and Riccati Equations General form of algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. 3/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

5 Introduction Large-Scale Algebraic Lyapunov and Riccati Equations General form of algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), 3/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

6 Introduction Large-Scale Algebraic Lyapunov and Riccati Equations General form of algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), 3/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

7 Introduction Large-Scale Algebraic Lyapunov and Riccati Equations General form of algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. 3/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

8 Introduction Large-Scale Algebraic Lyapunov and Riccati Equations General form of algebraic Riccati equation (ARE) for A, G = G T, W = W T R n n given and X R n n unknown: G = 0 = Lyapunov equation: 0 = R(X ) := A T X + XA XGX + W. 0 = L(X ) := A T X + XA + W. Typical situation in model reduction and optimal control problems for semi-discretized PDEs: n = (= unknowns!), A has sparse representation (A = M 1 S for FEM), G, W low-rank with G, W {BB T, C T C}, where B R n m, m n, C R p n, p n. Standard (eigenproblem-based) O(n 3 ) methods are not applicable! 3/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

9 Introduction Low-Rank Approximation Consider spectrum of ARE solution (analogous for Lyapunov equations). Example: Linear 1D heat equation with point control, Ω = [ 0, 1 ], FEM discretization using linear B-splines, h = 1/100 = n = /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

10 Introduction Low-Rank Approximation Consider spectrum of ARE solution (analogous for Lyapunov equations). Example: Linear 1D heat equation with point control, Ω = [ 0, 1 ], FEM discretization using linear B-splines, h = 1/100 = n = 101. Idea: X = X T 0 = X = ZZ T = n λ k z k zk T Z (r) (Z (r) ) T = k=1 r λ k z k zk T. k=1 = Goal: compute Z (r) R n r directly w/o ever forming X! 4/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

11 Introduction Review: LRCF-ADI for Lyapunov Equations Consider FX + XF T = GG T ADI iteration for the Lyapunov equation (LE) [Wachspress 95] For j = 1,..., J X 0 = 0 (F + p j I )X j 1 = GG T X 2 j 1 (F T p j I ) (F + p j I )Xj T = GG T X T (F T p j 1 j I ) 2 5/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

12 Introduction Review: LRCF-ADI for Lyapunov Equations Consider FX + XF T = GG T ADI iteration for the Lyapunov equation (LE) [Wachspress 95] For j = 1,..., J X 0 = 0 (F + p j I )X j 1 = GG T X 2 j 1 (F T p j I ) (F + p j I )Xj T = GG T X T (F T p j 1 j I ) 2 Rewrite as one step iteration and factorize X i = Z i Zi T, i = 0,..., J Z 0 Z0 T = 0 Z j Zj T = 2p j (F + p j I ) 1 GG T (F + p j I ) T +(F + p j I ) 1 (F p j I )Z j 1 Zj 1 T ji ) T (F + p j I ) T 5/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

13 Introduction Review: LRCF-ADI for Lyapunov Equations Z j = [ 2p j (F + p j I ) 1 G, (F + p j I ) 1 (F p j I )Z j 1 ] [Penzl 00] 6/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

14 Introduction Review: LRCF-ADI for Lyapunov Equations Z j = [ 2p j (F + p j I ) 1 G, (F + p j I ) 1 (F p j I )Z j 1 ] [Penzl 00] Observing that (F p i I ), (F + p k I ) 1 commute, we rewrite Z J as Z J = [z J, P J 1 z J, P J 2 (P J 1 z J ),..., P 1 (P 2 P J 1 z J )], where and P i := z J = 2p J (F + p J I ) 1 G 2pi 2pi+1 [ I (pi + p i+1 )(F + p i I ) 1]. [Li/White 02] 6/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

15 Introduction Review: LRCF-ADI for Lyapunov Equations Algorithm 1 Low-rank Cholesky factor ADI iteration (LRCF-ADI) [Penzl 97/ 00, Li/White 99/ 02, B./Li/Penzl 99/ 08] Input: F,G defining FX + XF T = GG T and shifts {p 1,..., p imax } Output: Z = Z imax C n t imax, such that ZZ H X 1: For V 1 solve (F + p 1 I ) V 1 = 2 Re (p 1 )G 2: Z 1 = V 1 3: for i = 2, 3,..., i max do 4: For Ṽ solve (F + p i I )Ṽ = V i 1 5: V i = ( ) Re (p i )/ Re (p i 1 ) V i 1 (p i + p i 1 )Ṽ 6: Z i = [Z i 1 V i ] 7: end for 7/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

16 Introduction Review: LRCF-ADI for Lyapunov Equations Algorithm 1 General. Low-rank Cholesky factor ADI iteration (G-LRCF-ADI) [B. 04, B./Saak 09, S. 09] Input: E,F,G defining FXE T +EXF T = GG T and shifts {p 1,..., p imax } Output: Z = Z imax C n t imax, such that ZZ H X 1: For V 1 solve (F + p 1 E) V 1 = 2 Re (p 1 )G 2: Z 1 = V 1 3: for i = 2, 3,..., i max do 4: For Ṽ solve (F + p i E)Ṽ = EV i 1 5: V i = ( ) Re (p i )/ Re (p i 1 ) V i 1 (p i + p i 1 )Ṽ 6: Z i = [Z i 1 V i ] 7: end for 7/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

17 Introduction Krylov Subspace Based Solvers for Lyapunov Equations Consider Schur/singular value decomposition X = UΣU T, U R n n, U T U = I, Σ = diag (σ 1,..., σ n ) and σ 1 σ 2 σ n. The best rank-m Frobenius-norm approximation to X is thus given by [ ] Σm 0 X m := U U T = U 0 0 m Σ m Um T. 8/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

18 Introduction Krylov Subspace Based Solvers for Lyapunov Equations Consider Schur/singular value decomposition X = UΣU T, U R n n, U T U = I, Σ = diag (σ 1,..., σ n ) and σ 1 σ 2 σ n. The best rank-m Frobenius-norm approximation to X is thus given by [ ] Σm 0 X m := U U T = U 0 0 m Σ m Um T. Krylov projection idea [Saad 90, Jaimoukha/Kasenally 94] Solve (Um T FU m )Y m + Y m (Um T F T U m ) = Um T GG T U m, (1) on colspan(u m ) and get X m as X m = U m Y m Um T. 8/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

19 Introduction Krylov Subspace Based Solvers for Lyapunov Equations Consider Schur/singular value decomposition X = UΣU T, U R n n, U T U = I, Σ = diag (σ 1,..., σ n ) and σ 1 σ 2 σ n. The best rank-m Frobenius-norm approximation to X is thus given by [ ] Σm 0 X m := U U T = U 0 0 m Σ m Um T. Note that a factorization Z m Z T m = X m can easily be computed from a Cholesky factorization of Y m = Z m Z T m as Z m = U m Zm. 8/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

20 Introduction Krylov Subspace Based Solvers for Lyapunov Equations Algorithm 2 Basic Krylov Subspace Method for the Lyapunov Equation Input: F,G defining FX + XF T = GG T, an initial Krylov subspace V, e.g., V = K p (F, G) with orthogonal basis V C n p. Output: Z C n t, such that ZZ H X repeat if not first step then increase dimension of V and update V. end if Solve the small LE for Z with a classical solver: (V T FV ) Z Z T + Z Z T (V T F T V ) = V T GG T V, Lift Z to the full space: Z = U Z until res(z)< TOL 9/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

21 LRCF-ADI with Galerkin-Projection-Acceleration ADI and Rational Krylov [Li 00; Theorem 2] interprets the column span of the ADI solution as a certain rational Krylov subspace 8 < 1 L(F, G, p) := span :..., Y (F + p i I ) 1 G,..., (F + p 2 I ) 1 (F + p 1 I ) 1 G, i= j (F + p 1 I ) 1 G, G, (F + p 1 I )G, 9 jy = (F + p 2 I )(F + p 1 I )G,..., (F + p i I )G... ; i=1 10/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

22 LRCF-ADI with Galerkin-Projection-Acceleration ADI and Rational Krylov [Li 00; Theorem 2] interprets the column span of the ADI solution as a certain rational Krylov subspace 8 < 1 L(F, G, p) := span :..., Y (F + p i I ) 1 G,..., (F + p 2 I ) 1 (F + p 1 I ) 1 G, i= j (F + p 1 I ) 1 G, G, (F + p 1 I )G, 9 jy = (F + p 2 I )(F + p 1 I )G,..., (F + p i I )G... ; i=1 Idea Solve on current subspace of L(F, G, p) in the ADI step to increase the quality of the iterate. 10/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

23 LRCF-ADI with Galerkin-Projection-Acceleration Projected ADI Step Projected ADI Step LRCF-ADI-GP [B./Li/Truhar 09, Saak 09, B./Saak 10] 1 Compute the LRCF-ADI iterate Z i 2 Compute orthogonal basis via QR factorization: Q i R i Π i = Z i a 3 Solve (for Z) the projected Lyapunov equation (Qi T FQ i ) Z Z T + Z Z T (Qi T F T Q i ) = Qi T GG T Q i 4 Update Z i according to Z i := Q i Z a economy size QR with column pivoting; crucial to compute correct subspace if Z i rank deficient. 11/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

24 LRCF-ADI with Galerkin-Projection-Acceleration Projected ADI Step Projected ADI Step LRCF-ADI-GP [B./Li/Truhar 09, Saak 09, B./Saak 10] 1 Compute the LRCF-ADI iterate Z i 2 Compute orthogonal basis via QR factorization: Q i R i Π i = Z i 3 Solve (for Z) the projected Lyapunov equation (Q T i FQ i ) Z Z T + Z Z T (Q T i F T Q i ) = Q T i GG T Q i 4 Update Z i according to Z i := Q i Z Need to ensure that projected systems remain stable, e.g., F + F T < 0 may perform projected ADI step only every k-th step (e.g. k = 5) restarted ADI with shifts Λ(Q T i FQ i ). 11/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

25 LRCF-ADI with Galerkin-Projection-Acceleration Projected ADI Step Projected ADI Step G-LRCF-ADI-GP [B./Li/Truhar 09, Saak 09, B./Saak 10] 1 Compute the G-LRCF-ADI iterate Z i 2 Compute orthogonal basis via QR factorization: Q i R i Π i = Z i 3 Solve (for Z) the projected Lyapunov equation (Q T i FQ i ) Z Z T (Q T i E T Q i ) + (Q T i EQ i ) Z Z T (Q T i F T Q i ) = Q T i GG T Q i 4 Update Z i according to Z i := Q i Z 11/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

26 LRCF-ADI with Galerkin-Projection-Acceleration Projected ADI Step F Z Z T + Z Z T F T = G G T Legend: new factor old factor original matrix original rhs projected matrix projected Cholesky factor projected rhs 12/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

27 LRCF-ADI with Galerkin-Projection-Acceleration Projected ADI Step F Z Z T + Z Z T F T = G G T F m Fm T G Gm T m Legend: new factor old factor original matrix original rhs projected matrix projected Cholesky factor projected rhs 12/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

28 LRCF-ADI with Galerkin-Projection-Acceleration Projected ADI Step F m Cm C T m + C T m F T m G m Cm = G T m Legend: new factor old factor original matrix original rhs projected matrix projected Cholesky factor projected rhs 12/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

29 LRCF-ADI with Galerkin-Projection-Acceleration Projected ADI Step F m Cm C T m + C T m F T m G m Cm = G T m F Z Z T + Z Z T F T = G G T Legend: new factor old factor original matrix original rhs projected matrix projected Cholesky factor projected rhs 12/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

30 LRCF-ADI with Galerkin-Projection-Acceleration Test Example: Optimal Cooling of Steel Profiles Mathematical model: boundary control for linearized 2D heat equation. c ρ t x = λ x, ξ Ω λ n x = κ(u k x), ξ Γ k, 1 k 7, x = 0, ξ Γ0. n = q = 7, p = FEM Discretization, different models for initial mesh (n = 371), 1, 2, 3, 4 steps of mesh refinement n = 1 357, 5 177, , Source: Physical model: courtesy of Mannesmann/Demag. Math. model: Tröltzsch/Unger 99/ 01, Penzl 99, S /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

31 LRCF-ADI with Galerkin-Projection-Acceleration Numerical Results steel profile n= good shifts 10 0 Iteration history for controllability gramian no projection every step every 5 steps 10 0 Iteration history for observability gramian no projection every step every 5 steps 10 2 normalized residual normalized residual iteration number iteration number 14/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

32 LRCF-ADI with Galerkin-Projection-Acceleration Numerical Results steel profile n= good shifts 100 Computation times time in seconds galerkin projection frequency 14/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

33 LRCF-ADI with Galerkin-Projection-Acceleration Numerical Results steel profile n= bad shifts 10 0 Iteration history for controllability gramian Iteration history for observability gramian normalized residual normalized residual no projection every step every 5 steps iteration number 10 6 no projection every step every 5 steps iteration number 15/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

34 LRCF-ADI with Galerkin-Projection-Acceleration Numerical Results steel profile n= bad shifts 2500 Computation times 2000 time in seconds galerkin projection frequency 15/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

35 LRCF-NM for the ARE 1 Introduction 2 LRCF-ADI with Galerkin-Projection-Acceleration 3 LRCF-NM for the ARE Newton s Method for AREs Low-Rank Newton-ADI (LRCF-NM) for AREs Test Examples Test Results (ADI-loop) Test Results (both-loops) Computation Time Scaling with Problem Size 16/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

36 LRCF-NM for the ARE Newton s Method for AREs Consider R(X ) := C T C + A T X + XA XBB T X = 0 Newton s Iteration for the ARE R X (N l ) = R(X l ), X l+1 = X l + N l, l = 0, 1,... where the Frechét derivative of R at X is the Lyapunov operator R X : Q (A BB T X ) T Q + Q(A BB T X ), i.e., in every Newton step solve a Lyapunov Equation [Kleinman 68] (A BB T X l ) T X l+1 + X l+1 (A BB T X l ) = C T C X l BB T X l. 17/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

37 LRCF-NM for the ARE Newton s Method for AREs Consider R(X ) := C T C + A T X + XA XBB T X = 0 Newton s Iteration for the ARE R X (N l ) = R(X l ), X l+1 = X l + N l, l = 0, 1,... where the Frechét derivative of R at X is the Lyapunov operator R X : Q (A BB T X ) T Q + Q(A BB T X ), i.e., in every Newton step solve a Lyapunov Equation [Kleinman 68] F T l X l+1 + X l+1 F l = G l G T l. 17/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

38 LRCF-NM for the ARE Newton s Method for AREs Consider R(X ) := C T C + A T X E + E T XA E T XBB T X E = 0 Newton s Iteration for the ARE R X (N l ) = R(X l ), X l+1 = X l + N l, l = 0, 1,... where the Frechét derivative of R at X is the Lyapunov operator R X : Q (A BB T X E) T QE + E T Q(A BB T X E), i.e., in every Newton step solve a Lyapunov Equation [Kleinman 68] F T l X l+1 E + E T X l+1 F l = G l G T l. 17/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

39 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Factored Newton-Kleinman Iteration [Benner/Li/Penzl 99/ 08] F l = A BB T X l =: A BK l G l = [C T Kl T ] is sparse + low rank is low rank factor 18/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

40 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Factored Newton-Kleinman Iteration [Benner/Li/Penzl 99/ 08] F l = A BB T X l =: A BK l G l = [C T Kl T ] apply LRCF-ADI in every Newton step is sparse + low rank is low rank factor exploit structure of F l using Sherman-Morrison-Woodbury formula 18/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

41 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Factored Newton-Kleinman Iteration [Benner/Li/Penzl 99/ 08] F l = A BB T X l E =: A BK l G l = [C T Kl T ] apply LRCF-ADI in every Newton step is sparse + low rank is low rank factor exploit structure of F l using Sherman-Morrison-Woodbury formula 18/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

42 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Algorithm 3 Low-Rank Cholesky Factor Newton Method (LRCF-NM) Input: A, B, C, K (0) for which A BK (0)T is stable Output: Z = Z (kmax ), such that ZZ H approximates the solution X of C T C + A T X + XA XBB T X = 0. 1: for k = 1, 2,..., k max do 2: Determine (sub)optimal ADI shift parameters p (k) 1, p(k) 2,... with respect to the matrix F (k) = A T K (k 1) B T. 3: G (k) = [ C T K (k 1) ] 4: Compute Z (k) using Algorithm 1 (LRCF-ADI) such that F (k) Z (k) Z (k)h + Z (k) Z (k)h F (k)t G (k) G (k)t. 5: K (k) = Z (k) (Z (k)h B) 6: end for 19/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

43 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Algorithm 3 Low-Rank Cholesky Factor Newton Method (G-LRCF-NM) Input: E, A, B, C, K (0) for which A BK (0)T is stable Output: Z = Z (kmax ), such that ZZ H approximates the solution X of C T C + A T X E + E T XA E T XBB T X E = 0. 1: for k = 1, 2,..., k max do 2: Determine (sub)optimal ADI shift parameters p (k) 1, p(k) 2,... with respect to the matrix F (k) = A T E T K (k 1) B T E T. 3: G (k) = [ C T K (k 1) ] 4: Compute Z (k) using Algorithm 1 (G-LRCF-ADI) such that F (k) Z (k) Z (k)h E + E T Z (k) Z (k)h F (k)t G (k) G (k)t. 5: K (k) = E T (Z (k) (Z (k)h B)) 6: end for 19/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

44 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Algorithm 3 Low-Rank Cholesky Factor Newton Method (LRCF-NM) Input: A, B, C, K (0) for which A BK (0)T is stable Output: Z = Z (kmax ), such that ZZ H approximates the solution X of C T C + A T X + XA XBB T X = 0. 1: for k = 1, 2,..., k max do 2: Determine (sub)optimal ADI shift parameters p (k) 1, p(k) 2,... with respect to the matrix F (k) = A T K (k 1) B T. 3: G (k) = [ C T K (k 1) ] 4: Compute Z (k) using Algorithm 1 (LRCF-ADI) or (LRCF-ADI-GP) such that F (k) Z (k) Z (k)h + Z (k) Z (k)h F (k)t G (k) G (k)t. 5: K (k) = Z (k) (Z (k)h B) 6: end for 20/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

45 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Algorithm 4 Simpl. Low-Rank Cholesky Factor Newton Method (LRCF-NM-S) Input: A, B, C, K (0) for which A BK (0)T is stable Output: Z = Z (kmax ), such that ZZ H approximates the solution X of C T C + A T X + XA XBB T X = 0. 1: Determine (sub)optimal ADI shift parameters p 1, p 2,... with respect to the matrix F (k) = A T K (0) B T. 2: for k = 1, 2,..., k max do 3: G (k) = [ C T K (k 1) ] 4: Compute Z (k) using Algorithm 1 (LRCF-ADI) or (LRCF-ADI-GP) such that F (k) Z (k) Z (k)h + Z (k) Z (k)h F (k)t G (k) G (k)t. 5: K (k) = Z (k) (Z (k)h B) 6: end for 20/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

46 LRCF-NM for the ARE Low-Rank Newton-ADI (LRCF-NM) for AREs Algorithm 5 Low-Rank Cholesky Factor Galerkin-Newton Method (LRCF-NM-GP) Input: A, B, C, K (0) for which A BK (0)T is stable Output: Z = Z (kmax ), such that ZZ H approximates the solution X of C T C + A T X + XA XBB T X = 0. 1: for k = 1, 2,..., k max do 2: Determine (sub)optimal ADI shift parameters p (k) 1, p(k) 2,... with respect to the matrix F (k) = A T K (k 1) B T. 3: G (k) = [ C T K (k 1) ] 4: Compute Z (k) using Algorithm 1 (LRCF-ADI) or (LRCF-ADI-GP) such that F (k) Z (k) Z (k)h + Z (k) Z (k)h F (k)t G (k) G (k)t. 5: Project ARE, solve and prolongate solution 6: K (k) = Z (k) (Z (k)h B) 7: end for 20/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

47 LRCF-NM for the ARE Test Examples Example 1: 3d Convection-Diffusion Equation FDM for 3D convection-diffusion equation on [0, 1] 3 proposed in [Simoncini 07], q = p = 1 non-symmetric A R n n, n = Example 2: 2d Convection-Diffusion Equation FDM for 2D convection-diffusion equations on [0, 1] 2 LyaPack benchmark, q = p = 1, e.g., demo l1 non-symmetric A R n n, n = shift parameters Penzl s heuristic from 50/25 Ritz/harmonic Ritz values of A 21/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

48 LRCF-NM for the ARE Test Results (ADI-loop): Example 1 Newton-ADI Newton-Galerkin-ADI LRCF-ADI-GP(5) NWT rel. change rel. residual ADI NWT rel. change rel. residual ADI CPU time: sec. CPU time: sec. test system: Intel Xeon GHz ; 16 GB RAM; 64Bit-MATLAB (R2010a) using threaded BLAS (romulus) stopping criterion tolerances: /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

49 LRCF-NM for the ARE Test Results (ADI-loop): Example 2 Newton-ADI Newton-Galerkin-ADI LRCF-ADI-GP(5) NWT rel. change rel. residual ADI NWT rel. change rel. residual ADI CPU time: sec. CPU time: sec. test system: Intel Core 2 Quad Q GHz; 4 GB RAM; 64Bit-MATLAB (R2009a) using threaded BLAS (reynolds) stopping criterion tolerances: /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

50 LRCF-NM for the ARE Test Results (both-loops): Example 1 Newton-ADI NWT rel. change rel. residual ADI CPU time: sec. NG-ADI inner= 5, outer= 1 NWT rel. change rel. residual ADI CPU time: sec. NG-ADI inner= 1, outer= 1 NWT rel. change rel. residual ADI CPU time: sec. NG-ADI inner= 0, outer= 1 NWT rel. change rel. residual ADI CPU time: sec. test system: Intel Xeon GHz ; 16 GB RAM; 64Bit-MATLAB (R2010a) using threaded BLAS (romulus) stopping criterion tolerances: /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

51 LRCF-NM for the ARE Test Results (both-loops): Example 2 Newton-ADI NWT rel. change rel. residual ADI CPU time: sec. NG-ADI inner= 5, outer= 1 NWT rel. change rel. residual ADI CPU time: 24.1 sec. NG-ADI inner= 1, outer= 1 NWT rel. change rel. residual ADI CPU time: 26.8 sec. NG-ADI inner= 0, outer= 1 NWT rel. change rel. residual ADI CPU time: 24.0 sec. test system: Intel Core 2 Quad Q GHz; 4 GB RAM; 64Bit-MATLAB (R2009a) using threaded BLAS (reynolds) stopping criterion tolerances: /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

52 LRCF-NM for the ARE Computation Time Scaling with Problem Size (0, 1) (1, 1) tx(ξ, t) = x(ξ, t) in Ω Ω Γ c νx = b(ξ) u(t) x νx = x on Γ c on Ω \ Γ c x(ξ, 0) = 1 (0, 0) (1, 0) Note: Here b(ξ) = 4 (1 ξ 2 ) ξ 2 for ξ Γ c and 0 otherwise, thus t R >0, we have u(t) R. 26/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

53 LRCF-NM for the ARE Computation Time Scaling with Problem Size (0, 1) (1, 1) tx(ξ, t) = x(ξ, t) in Ω Ω Γ c νx = b(ξ) u(t) x νx = x on Γ c on Ω \ Γ c x(ξ, 0) = 1 (0, 0) (1, 0) Note: Here b(ξ) = 4 (1 ξ 2 ) ξ 2 for ξ Γ c and 0 otherwise, thus t R >0, we have u(t) R. B h = M Γ,h b. 26/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

54 LRCF-NM for the ARE Computation Time Scaling with Problem Size (0, 1) (1, 1) Ω Γ c tx(ξ, t) = x(ξ, t) νx = b(ξ) u(t) x νx = x x(ξ, 0) = 1 in Ω on Γ c on Ω \ Γ c (0, 0) (1, 0) Consider: output equation y = Cx, where C : L 2 (Ω) x(ξ, t) R y(t) = x(ξ, t) dξ. Ω 26/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

55 LRCF-NM for the ARE Computation Time Scaling with Problem Size (0, 1) (1, 1) Ω Γ c tx(ξ, t) = x(ξ, t) νx = b(ξ) u(t) x νx = x x(ξ, 0) = 1 in Ω on Γ c on Ω \ Γ c (0, 0) (1, 0) Consider: output equation y = Cx, where C : L 2 (Ω) R x(ξ, t) y(t) = Ω x(ξ, t) dξ, C h = 1 M h. 26/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

56 LRCF-NM for the ARE Computation Time Scaling with Problem Size (0, 1) (1, 1) tx(ξ, t) = x(ξ, t) in Ω Ω Γ c νx = b(ξ) u(t) x νx = x on Γ c on Ω \ Γ c x(ξ, 0) = 1 (0, 0) (1, 0) Cost Function: J (u) = 0 y 2 (t) + u 2 (t) dt. 26/27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

57 LRCF-NM for the ARE Computation Time Scaling with Problem Size simplified Low Rank Newton-Galerkin ADI generalized state space form implementation Penzl shifts (16/50/25) with respect to initial matrices projection acceleration in every outer iteration step projection acceleration in every 5-th inner iteration step test system: Intel Xeon 3.00 GHz; 16 GB RAM; 64Bit-MATLAB (R2010a) using threaded BLAS (romulus) stopping criterion tolerances: /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

58 LRCF-NM for the ARE Computation Time Scaling with Problem Size Computation Times discretization level problem size time in seconds test system: Intel Xeon 3.00 GHz; 16 GB RAM; 64Bit-MATLAB (R2010a) using threaded BLAS (romulus) stopping criterion tolerances: /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs

59 LRCF-NM for the ARE Computation Time Scaling with Problem Size Scaling of CPU time time in seconds refinement level test system: Intel Xeon 3.00 GHz; 16 GB RAM; 64Bit-MATLAB (R2010a) using threaded BLAS (romulus) stopping criterion tolerances: /27 Peter Benner, Jens Saak Newton-Galerkin-ADI for AREs