Efficient Implementation of Large Scale Lyapunov and Riccati Equation Solvers

Similar documents
A Newton-Galerkin-ADI Method for Large-Scale Algebraic Riccati Equations

The Newton-ADI Method for Large-Scale Algebraic Riccati Equations. Peter Benner.

BALANCING-RELATED MODEL REDUCTION FOR DATA-SPARSE SYSTEMS

Model Reduction for Dynamical Systems

S N. hochdimensionaler Lyapunov- und Sylvestergleichungen. Peter Benner. Mathematik in Industrie und Technik Fakultät für Mathematik TU Chemnitz

MODEL REDUCTION BY A CROSS-GRAMIAN APPROACH FOR DATA-SPARSE SYSTEMS

Balanced Truncation Model Reduction of Large and Sparse Generalized Linear Systems

Passivity Preserving Model Reduction for Large-Scale Systems. Peter Benner.

Numerical Solution of Descriptor Riccati Equations for Linearized Navier-Stokes Control Problems

Accelerating Model Reduction of Large Linear Systems with Graphics Processors

BALANCING-RELATED MODEL REDUCTION FOR LARGE-SCALE SYSTEMS

Model Order Reduction of Continuous LTI Large Descriptor System Using LRCF-ADI and Square Root Balanced Truncation

Numerical solution of large-scale Lyapunov equations, Riccati equations, and linear-quadratic optimal control problems

Factorized Solution of Sylvester Equations with Applications in Control

A POD Projection Method for Large-Scale Algebraic Riccati Equations

In this article we discuss sparse matrix algorithms and parallel algorithms, Solving Large-Scale Control Problems

LYAPACK. A MATLAB Toolbox for Large Lyapunov and Riccati Equations, Model Reduction Problems, and Linear Quadratic Optimal Control Problems

Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix

arxiv: v1 [math.na] 4 Apr 2018

Max Planck Institute Magdeburg Preprints

Balancing-Related Model Reduction for Large-Scale Systems

Parallel Solution of Large-Scale and Sparse Generalized algebraic Riccati Equations

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)

BDF Methods for Large-Scale Differential Riccati Equations

Numerical Solution of Differential Riccati Equations on Hybrid CPU-GPU Platforms

Parallel Model Reduction of Large Linear Descriptor Systems via Balanced Truncation

Exploiting off-diagonal rank structures in the solution of linear matrix equations

Parametrische Modellreduktion mit dünnen Gittern

Order reduction numerical methods for the algebraic Riccati equation. V. Simoncini

Adaptive rational Krylov subspaces for large-scale dynamical systems. V. Simoncini

On the numerical solution of large-scale linear matrix equations. V. Simoncini

Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques

Krylov Subspace Type Methods for Solving Projected Generalized Continuous-Time Lyapunov Equations

Linear and Nonlinear Matrix Equations Arising in Model Reduction

Numerical Methods I Non-Square and Sparse Linear Systems

Incomplete Cholesky preconditioners that exploit the low-rank property

DELFT UNIVERSITY OF TECHNOLOGY

The rational Krylov subspace for parameter dependent systems. V. Simoncini

Numerical Solution of Matrix Equations Arising in Control of Bilinear and Stochastic Systems

Downloaded 05/27/14 to Redistribution subject to SIAM license or copyright; see

Model reduction of coupled systems

Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations

On the numerical solution of large-scale sparse discrete-time Riccati equations

Preconditioned inverse iteration and shift-invert Arnoldi method

Model order reduction of large-scale dynamical systems with Jacobi-Davidson style eigensolvers

Structure preserving Krylov-subspace methods for Lyapunov equations

On the numerical solution of large-scale sparse discrete-time Riccati equations CSC/ Chemnitz Scientific Computing Preprints

ADI-preconditioned FGMRES for solving large generalized Lyapunov equations - A case study

Accelerating Band Linear Algebra Operations on GPUs with Application in Model Reduction

SOLUTION OF ALGEBRAIC RICCATI EQUATIONS ARISING IN CONTROL OF PARTIAL DIFFERENTIAL EQUATIONS

Inexact inverse iteration with preconditioning

Solving linear systems (6 lectures)

Low Rank Solution of Data-Sparse Sylvester Equations

ON THE NUMERICAL SOLUTION OF LARGE-SCALE RICCATI EQUATIONS

Lecture 3: Inexact inverse iteration with preconditioning

The norms can also be characterized in terms of Riccati inequalities.

Chemnitz Scientific Computing Preprints

Model Reduction for Unstable Systems

A Continuation Approach to a Quadratic Matrix Equation

Numerical Solution of Optimal Control Problems for Parabolic Systems

J.I. Aliaga 1 M. Bollhöfer 2 A.F. Martín 1 E.S. Quintana-Ortí 1. March, 2009

Balancing-Related Model Reduction for Parabolic Control Systems

Sonderforschungsbereich 393

Order Reduction of a Distributed Parameter PEM Fuel Cell Model

Structured Krylov Subspace Methods for Eigenproblems with Spectral Symmetries

SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA

THE solution of the absolute value equation (AVE) of

SPARSE SOLVERS POISSON EQUATION. Margreet Nool. November 9, 2015 FOR THE. CWI, Multiscale Dynamics

FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION

On the -Sylvester Equation AX ± X B = C

Matrix Equations and and Bivariate Function Approximation

Algorithm 853: an Efficient Algorithm for Solving Rank-Deficient Least Squares Problems

FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION

Towards high performance IRKA on hybrid CPU-GPU systems

Accelerating linear algebra computations with hybrid GPU-multicore systems.

Numerical Linear Algebra

Program Lecture 2. Numerical Linear Algebra. Gaussian elimination (2) Gaussian elimination. Decompositions, numerical aspects

MPC/LQG for infinite-dimensional systems using time-invariant linearizations

Denis ARZELIER arzelier

Moving Frontiers in Model Reduction Using Numerical Linear Algebra

Lecture 11: CMSC 878R/AMSC698R. Iterative Methods An introduction. Outline. Inverse, LU decomposition, Cholesky, SVD, etc.

SYSTEM-THEORETIC METHODS FOR MODEL REDUCTION OF LARGE-SCALE SYSTEMS: SIMULATION, CONTROL, AND INVERSE PROBLEMS. Peter Benner

Direct Methods for Matrix Sylvester and Lyapunov Equations

Sparse BLAS-3 Reduction

MODERN CONTROL DESIGN

Direct and Incomplete Cholesky Factorizations with Static Supernodes

Scientific Computing

Math 671: Tensor Train decomposition methods II

Key words. preconditioned conjugate gradient method, saddle point problems, optimal control of PDEs, control and state constraints, multigrid method

A robust multilevel approximate inverse preconditioner for symmetric positive definite matrices

1 Implementation (continued)

Solving Large Scale Lyapunov Equations

Gramian-Based Model Reduction for Data-Sparse Systems CSC/ Chemnitz Scientific Computing Preprints

Boundary Value Problems and Iterative Methods for Linear Systems

AM205: Assignment 2. i=1

Direct solution methods for sparse matrices. p. 1/49

1258 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 23, NO. 3, AUGUST 2008

Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms

Incomplete LU Preconditioning and Error Compensation Strategies for Sparse Matrices

Transcription:

Efficient Implementation of Large Scale Lyapunov and Riccati Equation Solvers Jens Saak joint work with Peter Benner (MiIT) Professur Mathematik in Industrie und Technik (MiIT) Fakultät für Mathematik Technische Universität Chemnitz Computational Methods with Applications Harrachov August 24, 2007 1/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Aim of this talk What is the aim of this talk? Promote the upcoming release 1.1 of the LyaPack software package. 2/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Aim of this talk What is the aim of this talk? Promote the upcoming release 1.1 of the LyaPack software package. What is LyaPack? 2/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Aim of this talk What is the aim of this talk? Promote the upcoming release 1.1 of the LyaPack software package. What is LyaPack? Matlab toolbox for solving large scale Lyapunov equations (applications like in M. Embrees plenary talk on Tuesday) Riccati equations linear quadratic optimal control problems 2/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Origin of the Riccati equations semi discrete parabolic PDE output equation ẋ(t) = Ax(t) + Bu(t) x(0) = x 0 X. (Cauchy) y(t) = Cx(t) (output) cost function J (u) = 1 < y, y > + < u, u > dt (cost) 2 0 and the linear quadratic regulator problem is LQR problem Minimize the quadratic (cost) with respect to the linear constraints (Cauchy),(output). 3/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Origin of the Riccati equations semi discrete parabolic PDE output equation ẋ(t) = Ax(t) + Bu(t) x(0) = x 0 X. (Cauchy) y(t) = Cx(t) (output) cost function J (u) = 1 < Cx, Cx > + < u, u > dt (cost) 2 0 and the linear quadratic regulator problem is LQR problem Minimize the quadratic (cost) with respect to the linear constraints (Cauchy),(output). 3/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Origin of the Riccati equations In the open literature it is well understood that the optimal feedback control is given as u = B T X x, where X is the minimal, positive semidefinite, selfadjoint solution of the algebraic Riccati equation 0 = R(X ) := C T C + A T X + XA XBB T X. 4/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Outline LRCF Newton Method for the ARE 1 LRCF Newton Method for the ARE 2 3 4 Column Compression for the low rank factors 5 6 5/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations 1 LRCF Newton Method for the ARE Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations 2 3 4 Column Compression for the low rank factors 5 6 6/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Large Scale Riccati and Lyapunov Equations Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations We are interested in solving algebraic Riccati equations 0 = A T P + PA PBB T P + C T C =: R(P). (ARE) where A R n n sparse, n N large B R n m and m N with m n C R p n and p N with p n 7/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Large Scale Riccati and Lyapunov Equations Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations We are interested in solving algebraic Riccati equations 0 = A T P + PA PBB T P + C T C =: R(P). (ARE) where A R n n sparse, n N large B R n m and m N with m n and Lyapunov equations C R p n and p N with p n with F R n n sparse or sparse + low rank update, n N large F T P + PF = GG T. (LE) G R n m and m N with m n 7/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Newton s method for solving the ARE Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations Newton s iteration for the ARE R P (N l ) = R(P l ), P l+1 = P l + N l, where the Frechét derivative of R at P is the Lyapunov operator R P : Q (A BB T P) T Q + Q(A BB T P), can be rewritten as one iteration step (A BB T P l ) T P l+1 + P l+1 (A BB T P l ) = C T C P l BB T P l i.e. in every Newton step we have to solve a Lyapunov equation of the form (LE) 8/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Cholesky factor ADI for Lyapunov equations Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations Recall Peaceman Rachford ADI: Consider Au = s where A R n n spd, s R n. ADI Iteration Idea: Decompose A = H + V with H, V R n n such that can be solved easily/efficiently. (H + pi )v = r (V + pi )w = t 9/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Cholesky factor ADI for Lyapunov equations Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations Recall Peaceman Rachford ADI: Consider Au = s where A R n n spd, s R n. ADI Iteration Idea: Decompose A = H + V with H, V R n n such that can be solved easily/efficiently. ADI Iteration (H + pi )v = r (V + pi )w = t If H, V spd p j, j = 1, 2,...J such that u 0 = 0 (H + p j I )u j 1 = (p 2 j I V )u j 1 + s (PR-ADI) (V + p j I )u j = (p j I H)u j 1 + s 2 converges to u R n solving Au = s. 9/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Cholesky factor ADI for Lyapunov equations Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations The Lyapunov operator L : P F T P + PF can be decomposed into the linear operators L H : P F T P L V : P PF. Such that in analogy to (PR-ADI) we find the ADI iteration for the Lyapunov equation (LE) P 0 = 0 (F T + p j I )P j 1 = GG T P 2 j 1 (F p j I ) (F T + p j I )Pj T = GG T P T (F p j 1 j I ) 2 (LE-ADI) 10/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Cholesky factor ADI for Lyapunov equations Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations Remarks: If F is sparse or sparse + low rank update, i.e. F = A + VU T then F T + p j I can be written as à + UV T, where à = AT + p j I and its inverse can be expressed as (F T + p j I ) 1 = (à + UV T ) 1 = à 1 à 1 U(I + V T à 1 U) 1 V T à 1 by the Sherman-Morrison-Woodbury formula. 11/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Cholesky factor ADI for Lyapunov equations Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations Remarks: If F is sparse or sparse + low rank update, i.e. F = A + VU T then F T + p j I can be written as à + UV T, where à = AT + p j I and its inverse can be expressed as (F T + p j I ) 1 = (à + UV T ) 1 = à 1 à 1 U(I + V T à 1 U) 1 V T à 1 by the Sherman-Morrison-Woodbury formula. Note: We only need to be able to multiply with A, solve systems with A and solve shifted systems with A T + p j I 11/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Cholesky factor ADI for Lyapunov equations Remarks: Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations If F is sparse or sparse + low rank update, i.e. F = A + VU T then F T + p j I can be written as à + UV T, where à = AT + p j I and its inverse can be expressed as (F T + p j I ) 1 = (à + UV T ) 1 = à 1 à 1 U(I + V T à 1 U) 1 V T à 1 by the Sherman-Morrison-Woodbury formula. (LE-ADI) can be rewritten to iterate on the low rank Cholesky factors Z j of P j to exploit rk(p j ) n. [J. R. Li and J. White 2002; T. Penzl 1999; P. Benner, J.R. Li and T. Penzl 2000] 11/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Cholesky factor ADI for Lyapunov equations Remarks: Large Scale Riccati and Lyapunov Equations Newton s method for solving the ARE Cholesky factor ADI for Lyapunov equations If F is sparse or sparse + low rank update, i.e. F = A + VU T then F T + p j I can be written as à + UV T, where à = AT + p j I and its inverse can be expressed as (F T + p j I ) 1 = (à + UV T ) 1 = à 1 à 1 U(I + V T à 1 U) 1 V T à 1 by the Sherman-Morrison-Woodbury formula. (LE-ADI) can be rewritten to iterate on the low rank Cholesky factors Z j of P j to exploit rk(p j ) n. [J. R. Li and J. White 2002; T. Penzl 1999; P. Benner, J.R. Li and T. Penzl 2000] When solving (ARE) to compute the feedback in an LQR-problem for a semidiscretized parabolic PDE, the LRCF-Newton-ADI can directly iterate on the feedback matrix K R n p to save even more memory. [T. Penzl 1999; P. Benner, J. R. Li and T. Penzl 2000] 11/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Motivation 1 LRCF Newton Method for the ARE 2 Motivation 3 4 Column Compression for the low rank factors 5 6 12/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Motivation Use sparse direct solvers Store LU or Cholesky factors frequently used (e.g. for M or A + p j I in case of cyclically used shifts). Save storage by reordering Upcoming LyaPack 1.1 let s you choose between: symmetric reverse Cuthill-McKee (RCM 1 ) reordering approximate minimum degree (AMD 2 ) reordering symmetric AMD 2 1 [A. George and J. W.-H. Liu 1981] 2 [P. Amestoy, T. A. Davis, and I. S. Duff 1996.] 13/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE Motivation Motivation Motivating example: Mass matrix M from a FEM semidiscretization of a 2d heat equation. Dimension of the discrete system: 1357 M MRCM 0 200 400 MAMD 0 0 200 200 400 400 600 600 600 800 800 800 1000 1000 1000 1200 1200 0 200 400 600 800 1000 1200 1200 0 200 400 nz = 8997 14/26 jens.saak@mathematik.tu-chemnitz.de 600 800 nz = 8997 Jens Saak 1000 1200 0 200 400 600 800 nz = 8997 Efficient Large Scale Lyapunov and ARE Solvers 1000 1200

LRCF Newton Method for the ARE Motivation Motivation Motivating example: Mass matrix M from a FEM semidiscretization of a 2d heat equation. Dimension of the discrete system: 1357 M MRCM 0 200 400 MAMD 0 0 200 200 400 400 600 600 600 800 800 800 1000 1000 1000 1200 1200 0 200 400 600 800 1000 1200 0 1200 200 400 nz = 8997 800 1000 1200 400 600 800 1000 0 0 200 200 400 400 600 600 800 800 1000 1000 0 0 200 400 600 800 1000 200 400 600 800 1200 nz = 32890 nz = 112423 jens.saak@mathematik.tu-chemnitz.de 200 400 600 800 1000 1200 1000 1200 chol(mamd) 1200 1200 1200 0 nz = 8997 chol(mrcm) chol(m) 0 200 14/26 600 nz = 8997 Jens Saak 1000 1200 0 200 400 600 800 nz = 11775 Efficient Large Scale Lyapunov and ARE Solvers

Available (sub)optimal choices Numerical Tests 1 LRCF Newton Method for the ARE 2 3 Available (sub)optimal choices Numerical Tests 4 Column Compression for the low rank factors 5 6 15/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Available (sub)optimal choices Numerical Tests Optimal parameters solve the min-max-problem min {p j j=1,...,j} R max γ σ(f ) J (p j λ) (p j + λ). j=1 Remark Also known as rational Zolotarev problem since he solved it first on real intervals enclosing the spectra in 1877. Another solution for the real case was presented by Wachspress/Jordan in 1963. 16/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Available (sub)optimal choices Numerical Tests Optimal parameters solve the min-max-problem min {p j j=1,...,j} R max γ σ(f ) J (p j λ) (p j + λ). j=1 Remark Wachspress and Starke presented different strategies to compute suboptimal shifts for the complex case around 1990. Wachspress: elliptic Integral evaluation based shifts Starke: Leja Point based shifts (recall M. Embrees plenary talk on Tuesday) 16/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Available (sub)optimal choices Available (sub)optimal choices Numerical Tests ADI shift parameter choices in upcoming LyaPack 1.1 1 heuristic parameters [T. Penzl 1999] use selected Ritz values as shifts suboptimal convergence might be slow in general complex for complex spectra 2 approximate Wachspress parameters [P. Benner, H. Mena, J. Saak 2006] optimal for real spectra parameters real if imaginary parts are small good approximation of the outer spectrum of F needed possibly expensive computation 3 only real heuristic parameters avoids complex computation and storage requirements can be slow if many Ritz values are complex 4 real parts of heuristic parameters avoids complex computation and storage requirements suitable if imaginary parts are small 17/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Available (sub)optimal choices Available (sub)optimal choices Numerical Tests ADI shift parameter choices in upcoming LyaPack 1.1 1 heuristic parameters [T. Penzl 1999] use selected Ritz values as shifts suboptimal convergence might be slow in general complex for complex spectra 2 approximate Wachspress parameters [P. Benner, H. Mena, J. Saak 2006] optimal for real spectra parameters real if imaginary parts are small good approximation of the outer spectrum of F needed possibly expensive computation 3 only real heuristic parameters avoids complex computation and storage requirements can be slow if many Ritz values are complex 4 real parts of heuristic parameters avoids complex computation and storage requirements suitable if imaginary parts are small 17/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Available (sub)optimal choices Available (sub)optimal choices Numerical Tests ADI shift parameter choices in upcoming LyaPack 1.1 1 heuristic parameters [T. Penzl 1999] use selected Ritz values as shifts suboptimal convergence might be slow in general complex for complex spectra 2 approximate Wachspress parameters [P. Benner, H. Mena, J. Saak 2006] optimal for real spectra parameters real if imaginary parts are small good approximation of the outer spectrum of F needed possibly expensive computation 3 only real heuristic parameters avoids complex computation and storage requirements can be slow if many Ritz values are complex 4 real parts of heuristic parameters avoids complex computation and storage requirements suitable if imaginary parts are small 17/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Available (sub)optimal choices Available (sub)optimal choices Numerical Tests ADI shift parameter choices in upcoming LyaPack 1.1 1 heuristic parameters [T. Penzl 1999] use selected Ritz values as shifts suboptimal convergence might be slow in general complex for complex spectra 2 approximate Wachspress parameters [P. Benner, H. Mena, J. Saak 2006] optimal for real spectra parameters real if imaginary parts are small good approximation of the outer spectrum of F needed possibly expensive computation 3 only real heuristic parameters avoids complex computation and storage requirements can be slow if many Ritz values are complex 4 real parts of heuristic parameters avoids complex computation and storage requirements suitable if imaginary parts are small 17/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Test example Available (sub)optimal choices Numerical Tests Centered finite difference discretized 2d convection diffusion equation: ẋ = x 10x x 100x y + b(x, y)u(t) on the unit square with Dirichlet boundary conditions. (demo l1.m) 18/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Test example Available (sub)optimal choices Numerical Tests Centered finite difference discretized 2d convection diffusion equation: ẋ = x 10x x 100x y + b(x, y)u(t) on the unit square with Dirichlet boundary conditions. (demo l1.m) grid size: 75 75 #states = 5625 #unknowns in X = 5625 2 32 10 6 Computations carried out on Intel Core2 Duo @2.13GHz Cache: 2048kB RAM: 2GB 18/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Test example Available (sub)optimal choices Numerical Tests Centered finite difference discretized 2d convection diffusion equation: ẋ = x 10x x 100x y + b(x, y)u(t) on the unit square with Dirichlet boundary conditions. (demo l1.m) grid size: 75 75 #states = 5625 #unknowns in X = 5625 2 32 10 6 heuristic parameters time: 44s residual norm: 1.0461e-11 heuristic real parts time: 13s residual norm: 9.0846e-12 appr. Wachspress time: 16s residual norm: 5.3196e-12 Remark heuristic parameters are complex problem size exceeds memory limitations in complex case Computations carried out on Intel Core2 Duo @2.13GHz Cache: 2048kB RAM: 2GB 18/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Column Compression for the low rank factors 1 LRCF Newton Method for the ARE 2 3 4 Column Compression for the low rank factors Numerical Tests 5 6 19/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Column Compression for the low rank factors Problem Low rank factors Z of the solutions X grow rapidly, since a constant number of columns is added in every ADI step. If convergence is weak, at some point #columns in Z > rk(z). Idea [Antoulas, Gugercin, Sorensen 2003] Use sequential Karhunen-Loeve algorithm; see [Baker 2004] uses QR + SVD for rank truncation 20/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Column Compression for the low rank factors Problem Low rank factors Z of the solutions X grow rapidly, since a constant number of columns is added in every ADI step. If convergence is weak, at some point #columns in Z > rk(z). Cheaper idea: Column compression using rank revealing QR factorization (RRQR) Consider X = ZZ T and rk(z) = r. Compute the RRQR 3 of Z [ ] Z T R11 R = QRΠ where R = 12 and R 0 R 11 R r r 22 now set Z T = [R 11 R 12 ] Π T then Z Z T =: X = X. 3 [ C.H. Bischof and G. Quintana-Ortí 1998] 20/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Column Compression for the low rank factors Numerical Tests Test example Centered finite difference discretized 2d convection diffusion equation: ẋ = x 10x x 100x y + b(x, y)u(t) on the unit square with Dirichlet boundary conditions. (demo l1.m) 21/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Column Compression for the low rank factors Numerical Tests Test example Centered finite difference discretized 2d convection diffusion equation: ẋ = x 10x x 100x y + b(x, y)u(t) on the unit square with Dirichlet boundary conditions. (demo l1.m) grid size: 75 75 #states = 5625 #unknowns in X = 5625 2 32 10 6 truncation TOL # col. in LRCF time res. norm 47 13s 9.0846e-12 eps 46 14s 1.9516e-11 eps 28 13s 1.9924e-11 Computations carried out on Intel Core2 Duo @2.13GHz Cache: 2048kB RAM: 2GB 21/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Numerical Tests Column Compression for the low rank factors Numerical Tests Test example Centered finite difference discretized 2d convection diffusion equation: ẋ = x 10x x 100x y + b(x, y)u(t) on the unit square with Dirichlet boundary conditions. (demo l1.m) grid size: 75 75 #states = 5625 #unknowns in X = 5625 2 32 10 6 Observation truncation TOL # col. in LRCF time res. norm 47 13s 9.0846e-12 eps 46 14s 1.9516e-11 eps 28 13s 1.9924e-11 [Benner and Quintana-Ortí 2005] showed that truncation tolerance eps in the low rank factor Z is sufficient to achieve an error eps in the solution X. Computations carried out on Intel Core2 Duo @2.13GHz Cache: 2048kB RAM: 2GB 21/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

LRCF Newton Method for the ARE 2 Basic Ideas in Contrast 1 LRCF Newton Method for the ARE 2 3 4 Column Compression for the low rank factors 5 2 Basic Ideas in Contrast 6 22/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

2 Basic Ideas in Contrast LRCF Newton Method for the ARE 2 Basic Ideas in Contrast Current Method Transform Mẋ = Ax + Bu y = Cx to x = Ã x + Bu y = C x where M = M L M U and x = M U x, Ã = M 1 L AM 1 U, B = M 1 B, C = CM 1 L U. - 2 additional sparse triangular solves in every multiplication with A - 2 additional sparse matrix vector multiplies in solution of Ãx = b and (Ã + p ji )x = b - B and C are dense even if B and C are sparse. + preserves symmetry if M, A are symmetric. 23/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

2 Basic Ideas in Contrast LRCF Newton Method for the ARE 2 Basic Ideas in Contrast Alternative Method Transform Mẋ = Ax + Bu y = Cx where à = M 1 A and B = M 1 B to ẋ = Ãx + Bu y = Cx + state variable untouched solution to (ARE), (LE) not transformed + exploiting pencil structure in (à + p j I ) = M 1 (A + p j M) reduces overhead - current user supplied function structure inefficient here rewrite of LyaPack kernel routines needed (work in progress) 23/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

2 Basic Ideas in Contrast LRCF Newton Method for the ARE 2 Basic Ideas in Contrast Alternative Method Transform Mẋ = Ax + Bu y = Cx where à = M 1 A and B = M 1 B to ẋ = Ãx + Bu y = Cx + state variable untouched solution to (ARE), (LE) not transformed + exploiting pencil structure in (à + p j I ) 1 = (A + p j M) 1 M reduces overhead - current user supplied function structure inefficient here rewrite of LyaPack kernel routines needed (work in progress) 23/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Conlusions Outlook 1 LRCF Newton Method for the ARE 2 3 4 Column Compression for the low rank factors 5 6 Conlusions Outlook 24/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Conlusions Conlusions Outlook Reordering strategies can reduce memory requirements by far 25/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Conlusions Conlusions Outlook Reordering strategies can reduce memory requirements by far new shift parameter selection allows easy improvements in ADI performance 25/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Conlusions Conlusions Outlook Reordering strategies can reduce memory requirements by far new shift parameter selection allows easy improvements in ADI performance Column compression via RRQR also drastically reduces storage requirements. 25/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Conlusions Conlusions Outlook Reordering strategies can reduce memory requirements by far new shift parameter selection allows easy improvements in ADI performance Column compression via RRQR also drastically reduces storage requirements. Especially helpful in differential Riccati equation solvers where 1 ARE solution needs to be stored in every step. 25/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Conlusions Conlusions Outlook Reordering strategies can reduce memory requirements by far new shift parameter selection allows easy improvements in ADI performance Column compression via RRQR also drastically reduces storage requirements. Especially helpful in differential Riccati equation solvers where 1 ARE solution needs to be stored in every step. Optimized treatment of generalized systems is work in progress 25/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Outlook Conlusions Outlook Theoretical Outlook Improve stopping Criteria for the ADI process. e.g. inside the LRCF-Newton method by interpretation as inexact Newton method following the ideas of Sachs et al. Optimize truncation tolerances for the RRQR Investigate dependence of residual errors in X on the truncation tolerance Stabilizing initial feedback computation Investigate whether the method in [K. Gallivan, X. Rao and P. Van Dooren 2006] can be implemented exploiting sparse matrix arithmetics. 26/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Outlook Conlusions Outlook Implementation TODOs User supplied functions for B (and C?) Introduce solvers for DREs Initial stabilizing feedback computation Improve handling of generalized systems of the form Mẋ = Ax + Bu. Improve the current Arnoldi implementation inside the heuristic ADI Parameter computation RRQR and column compression for complex factors. Simplify calling sequences, i.e. shorten commands by grouping parameters in structures Improve overall performance... 26/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers

Outlook Implementation TODOs Conlusions Outlook User supplied functions for B (and C?) Introduce solvers for DREs Initial stabilizing feedback computation Improve handling of generalized systems of the form Mẋ = Ax + Bu. Improve the current Arnoldi implementation inside the heuristic ADI Parameter computation RRQR and column compression for complex factors. Simplify calling sequences, i.e. shorten commands by grouping parameters in structures Thank you for your attention! Improve overall performance... 26/26 jens.saak@mathematik.tu-chemnitz.de Jens Saak Efficient Large Scale Lyapunov and ARE Solvers