Computational methods for large-scale matrix equations and application to PDEs. V. Simoncini

Transcription

1 Computational methods for large-scale matrix equations and application to PDEs V. Simoncini Dipartimento di Matematica, Università di Bologna 1

2 Some matrix equations Sylvester matrix equation AX+XB +D = 0 Eigenvalue pbs and tracking, Control, MOR, Assignment pbs, Riccati eqn Lyapunov matrix equation AX+XA +D = 0, D = D Stability analysis in Control and Dynamical systems, Signal processing, eigenvalue computations Multiterm matrix equation A 1 XB 1 +A 2 XB A l XB l = C (Stochastic) PDEs Focus: All or some of the matrices are large (and possibly sparse) 2

3 Some matrix equations Sylvester matrix equation AX+XB +D = 0 Eigenvalue pbs and tracking, Control, MOR, Assignment pbs, Riccati eqn Lyapunov matrix equation AX+XA +D = 0, D = D Stability analysis in Control and Dynamical systems, Signal processing, eigenvalue computations Multiterm matrix equation A 1 XB 1 +A 2 XB A l XB l = C (Stochastic) PDEs Focus: All or some of the matrices are large (and possibly sparse) 3

4 Some matrix equations Sylvester matrix equation AX+XB +D = 0 Eigenvalue pbs and tracking, Control, MOR, Assignment pbs, Riccati eqn Lyapunov matrix equation AX+XA +D = 0, D = D Stability analysis in Control and Dynamical systems, Signal processing, eigenvalue computations Multiterm matrix equation A 1 XB 1 +A 2 XB A l XB l = C Control, (Stochastic) PDEs,... Focus: All or some of the matrices are large (and possibly sparse) 4

5 Some matrix equations Sylvester matrix equation AX+XB +D = 0 Eigenvalue pbs and tracking, Control, MOR, Assignment pbs, Riccati eqn Lyapunov matrix equation AX+XA +D = 0, D = D Stability analysis in Control and Dynamical systems, Signal processing, eigenvalue computations Multiterm matrix equation A 1 XB 1 +A 2 XB A l XB l = C Control, (Stochastic) PDEs,... Focus: All or some of the matrices are large (and possibly sparse) 5

6 Solving the Lyapunov equation. The problem Approximate X in: AX+XA +BB = 0 A R n n neg.real B R n p, 1 p n Time-invariant linear dynamical system: x (t) = Ax(t)+Bu(t), x(0) = x 0 Closed form solution: X = 0 e ta BB e ta dt = 0 xx dt with x = exp( ta)b. X symmetric semidef. see, e.g., Antoulas 05, Benner 06 6

7 Solving the Lyapunov equation. The problem Approximate X in: AX+XA +BB = 0 A R n n neg.real B R n p, 1 p n Time-invariant linear system: x (t) = Ax(t)+Bu(t), x(0) = x 0 Closed form solution: X = 0 e ta BB e ta dt X symmetric semidef. see, e.g., Antoulas 05, Benner 06 7

8 Linear systems vs linear matrix equations Large linear systems: Ax = b, A R n n Krylov subspace methods (CG, MINRES, GMRES, BiCGSTAB, etc.) Preconditioners: find P such that AP 1 x = b x = P 1 x is easier and fast to solve Large linear matrix equation: AX+XA +BB = 0 No preconditioning to preserve symmetry X is a large, dense matrix low rank approximation X X = ZZ, Ztall 8

9 Linear systems vs linear matrix equations Large linear systems: Ax = b, A R n n Krylov subspace methods (CG, MINRES, GMRES, BiCGSTAB, etc.) Preconditioners: find P such that AP 1 x = b x = P 1 x is easier and fast to solve Large linear matrix equations: AX+XA +BB = 0 No preconditioning - to preserve symmetry X is a large, dense matrix low rank approximation X X = ZZ, Ztall 9

10 Linear systems vs linear matrix equations Large linear systems: Ax = b, A R n n Krylov subspace methods (CG, MINRES, GMRES, BiCGSTAB, etc.) Preconditioners: find P such that AP 1 x = b x = P 1 x is easier and fast to solve Large linear matrix equations: AX+XA +BB = 0 Kronecker formulation: (A I +I A)x = b x = vec(x) 10

11 Given an approximation space K, Projection-type methods X X m col(x m ) K Galerkin condition: R := AX m +X m A +BB K VmRV m = 0 K = Range(V m ) Assume VmV m = I m and let X m := V m Y m Vm. Projected Lyapunov equation: Vm(AV m Y m Vm +V m Y m VmA + BB )V m = 0 (VmAV m )Y m +Y m (VmA V m ) + VmBB V m = 0 Early contributions: Saad 90, Jaimoukha & Kasenally 94, for K = K m (A,B) = Range([B,AB,...,A m 1 B]) 11

12 Given an approximation space K, Projection-type methods X X m col(x m ) K Galerkin condition: R := AX m +X m A +BB K VmRV m = 0 K = Range(V m ) Assume VmV m = I m and let X m := V m Y m Vm. Projected Lyapunov equation: Vm(AV m Y m Vm +V m Y m VmA +BB )V m = 0 (VmAV m )Y m +Y m (VmA V m )+VmBB V m = 0 Early contributions: Saad 90, Jaimoukha & Kasenally 94, for K = K m (A,B) = Range([B,AB,...,A m 1 B]) 12

13 Given an approximation space K, Projection-type methods X X m col(x m ) K Galerkin condition: R := AX m +X m A +BB K VmRV m = 0 K = Range(V m ) Assume VmV m = I m and let X m := V m Y m Vm. Projected Lyapunov equation: Vm(AV m Y m Vm +V m Y m VmA +BB )V m = 0 (VmAV m )Y m +Y m (VmA V m )+VmBB V m = 0 Early contributions: Saad 90, Jaimoukha & Kasenally 94, for K = K m (A,B) = Range([B,AB,...,A m 1 B]) 13

14 Given an approximation space K, Projection-type methods X X m col(x m ) K Galerkin condition: R := AX m +X m A +BB K VmRV m = 0 K = Range(V m ) Assume VmV m = I m and let X m := V m Y m Vm. Projected Lyapunov equation: Vm(AV m Y m Vm +V m Y m VmA +BB )V m = 0 (VmAV m )Y m +Y m (VmA V m )+VmBB V m = 0 Early contributions: Saad 90, Jaimoukha & Kasenally 94, for K = K m (A,B) = Range([B,AB,...,A m 1 B]) 14

15 More recent options as approximation space Enrich space to decrease space dimension Extended Krylov subspace K = K m (A,B)+K m (A 1,A 1 B), that is, K = Range([B,A 1 B,AB,A 2 B,A 2 B,A 3 B,...,]) (Druskin & Knizhnerman 98, Simoncini 07) Rational Krylov subspace K = K := Range([B,(A s 1 I) 1 B,...,(A s m I) 1 B]) usually, {s 1,...,s m } C + chosen either a-priori or dynamically In both cases, for Range(V m ) = K, projected Lyapunov equation: (VmAV m )Y m +Y m (VmA V m )+VmBB V m = 0 X m = V m Y m Vm 15

16 More recent options as approximation space Enrich space to decrease space dimension Extended Krylov subspace K = K m (A,B)+K m (A 1,A 1 B), that is, K = Range([B,A 1 B,AB,A 2 B,A 2 B,A 3 B,...,]) (Druskin & Knizhnerman 98, Simoncini 07) Rational Krylov subspace K = K := Range([B,(A s 1 I) 1 B,...,(A s m I) 1 B]) usually, {s 1,...,s m } C + chosen either a-priori or dynamically In both cases, for Range(V m ) = K, projected Lyapunov equation: (VmAV m )Y m +Y m (VmA V m )+VmBB V m = 0 X m = V m Y m Vm 16

17 More recent options as approximation space Enrich space to decrease space dimension Extended Krylov subspace K = K m (A,B)+K m (A 1,A 1 B), that is, K = Range([B,A 1 B,AB,A 2 B,A 2 B,A 3 B,...,]) (Druskin & Knizhnerman 98, Simoncini 07) Rational Krylov subspace K = K := Range([B,(A s 1 I) 1 B,...,(A s m I) 1 B]) usually, {s 1,...,s m } C + chosen either a-priori or dynamically In both cases, for Range(V m ) = K, projected Lyapunov equation: (VmAV m )Y m +Y m (VmA V m )+VmBB V m = 0 X m = V m Y m Vm 17

18 Multiterm linear matrix equation A 1 XB 1 +A 2 XB A l XB l = C Applications: Control (Stochastic) PDEs Matrix least squares... Main device: Kronecker formulation ( B 1 A B l A l ) x = c Iterative methods: matrix-matrix multiplications and rank truncation (Benner, Breiten, Bouhamidi, Chehab, Damm, Grasedyck, Jbilou, Kressner, Matthies, Onwunta, Raydan, Stoll, Tobler, Zander, and many others...) 18

19 Multiterm linear matrix equation A 1 XB 1 +A 2 XB A l XB l = C Applications: Control (Stochastic) PDEs Matrix least squares... Main device: Kronecker formulation ( B 1 A B l A l ) x = c Iterative methods: matrix-matrix multiplications and rank truncation (Benner, Breiten, Bouhamidi, Chehab, Damm, Grasedyck, Jbilou, Kressner, Matthies, Onwunta, Raydan, Stoll, Tobler, Zander, and many others...) 19

20 Multiterm linear matrix equation A 1 XB 1 +A 2 XB A l XB l = C Applications: Control (Stochastic) PDEs Matrix least squares... Alternative approaches: low-rank approx in the problem space. Some examples: - Control problem - PDEs on uniform discretizations - Stochastic PDE 20

21 PDEs on uniform grids and separable coeffs ε u+φ 1 (x)ψ 1 (y)u x +φ 2 (x)ψ 2 (y)u y +γ 1 (x)γ 2 (y)u = f (x,y) Ω φ i,ψ i,γ i, i = 1,2 sufficiently regular functions + b.c. Problem discretization by means of a tensor basis Multiterm linear equation: εt 1 U εut 2 +Φ 1 B 1 UΨ 1 +Φ 2 UB 2 Ψ 2 +Γ 1 UΓ 2 = F Finite Diff.: U i,j = U(x i,y j ) approximate solution at the nodes 21

22 PDEs with random inputs Stochastic steady-state diffusion eqn: Find u : D Ω R s.t. P-a.s., (a(x,ω) u(x,ω)) = f(x) in D u(x,ω) = 0 on D f: deterministic; a: random field, linear function of finite no. of real-valued random variables ξ r : Ω Γ r R Common choice: truncated Karhunen Loève (KL) expansion, a(x,ω) = µ(x)+σ m λr φ r (x)ξ r (ω), r=1 µ(x): expected value of diffusion coef. σ: std dev. (λ r,φ r (x)) eigs of the integral operator V wrto V(x,x ) = 1 σ 2 C(x,x ) (λ r ց C : D D R covariance fun. ) 22

23 Discretization by stochastic Galerkin Approx with space in tensor product form a X h S p Ax = b, A = G 0 K 0 + m G r K r, b = g 0 f 0, r=1 x: expansion coef. of approx to u in the tensor product basis {ϕ i ψ k } K r R n x n x, FE matrices (sym) G r R n ξ n ξ, r = 0,1,...,m Galerkin matrices associated w/ S p (sym.) g 0 : first column of G 0 f 0 : FE rhs of deterministic PDE n ξ = dim(s p ) = (m+p)! m!p! n x n ξ huge a S p set of multivariate polyn of total degree p 23

24 The matrix equation formulation (G 0 K 0 +G 1 K G m K m )x = g 0 f 0 transforms into K 0 XG 0 +K 1 XG K m XG m = F, F = f 0 g0 (G 0 = I) Solution strategy. Conjecture: {K r } from trunc d Karhunen Loève (KL) expansion X X low rank, X = X1 X T 2 (Possibly extending results of Gradesyk, 2004) 24

25 Matrix Galerkin approximation of the deterministic part. 1 Approximation space K k and basis matrix V k : X X k = V k Y V k R k = 0, R k := K 0 X k +K 1 X k G K m X k G m f 0 g 0 Computational challenges: Generation of K k involved m+1 different matrices {K r }! Matrices K r have different spectral properties n x,n ξ so large that X k,r k should not be formed! (Powell & Silvester & Simoncini, SISC 2017) 25

26 Example. (a u) = 1, D = ( 1,1) 2. KL expansion. µ = 1, ξ r U( 3, ( ) 3) and C( x 1, x 2 ) = σ 2 exp x 1 x 2 1, n 2 x = 65,025, σ = 0.3 m p n ξ k inner n k rank time CG its K k X secs time (its) (8) (10) 87% (12) 5 1, (13) (8) (10) 89% 4 1, (12) 5 6, (13) (8) , (10) 93% 4 10, Out of Mem % of variance integral of a 26

27 Bilinear systems of matrix equations Find X R n 1 n 2 and P R m n 2 such that A 1 X+XA 2 +B T P = F 1 BX = F 2 with A i R n i n i, B R m n 1, F 1 R n 1 n 2, F 2 R m n 2, m n 1 Emerging matrix formulation of different application problems Constraint control Mixed formulations of stochastic diffusion problems Discretized deterministic/stochastic (Navier-)Stokes equations... 27

28 An example. Mixed FE formulation of stochastic Galerkin diffusion pb c 1 u p = 0, u = f, l Assume that c 1 = c 0 + λr c r ( x)ξ r (ω) and that an appropriate class of r=1 finite elements is used for the discretization of the problem (see, e.g., the derivation in Elman-Furnival-Powell, 2010) After discretization the problem reads: For l = 1 we obtain l G 0 K 0 + λgr K r G T 0 BT 0 u r=1 = p G 0 B 0 0 f K 0 XG 0 +K 1 XG 1 +B T 0 PG 0 = 0,B 0 XG 0 = F 28

29 An example. Mixed FE formulation of stochastic Galerkin diffusion pb c 1 u p = 0, u = f, l Assume that c 1 = c 0 + λr c r ( x)ξ r (ω) and that an appropriate class of r=1 finite elements is used for the discretization of the problem (see, e.g., the derivation in Elman-Furnival-Powell, 2010) After discretization the problem reads: For l = 1 we obtain l G 0 K 0 + λgr K r G T 0 BT 0 u r=1 = p G 0 B 0 0 f K 0 XG 0 +K 1 XG 1 +B T 0 PG 0 = 0, B 0 XG 0 = F 29

30 The bilinear case. Computational strategies A 1 X+XA 2 +B T P = F 1 BX = F 2 Kronecker formulation: A BT B O x = p f 1, A = I A 1 +A T 2 I, B = B I f 2 with x = vec(x), p = vec(p), f 1 = vec(f 1 ) and f 2 = vec(f 2 ) Extremely rich literature from saddle point algebraic linear systems Problem: Coefficient matrix has size (n 1 n 2 +mn 2 ) (n 1 n 2 +mn 2 ) 30

31 The bilinear case. Computational strategies. Cont d A 1 X+XA 2 +B T P = F 1 BX = F 2 Derive numerical strategies that directly work with the matrix equations: Small scale: Null space method Small and medium scale: Schur complement method (also directly applicable to trilinear case) Large scale: Iterative method for low rank F i, i = 1,2 Small and medium scale actually means Large scale for the Kronecker form! 31

32 Large scale problem. Iterative method. 1/3 A 1 X+XA 2 +B T P = F 1 BX = F 2 Rewrite as B A 1 B 0 with X + P I n 1 0 X A 2 = 0 0 P F 1, MZ+D 0 ZA 2 = F F 2 M,D 0 R (n 1+m) (n 1 +m) A 2 R n 2 n 2 nonsingular D 0 highly singular If F low rank, exploit projection-type strategies for Sylvester equations 32

33 Large scale problem. Iterative method. 2/3 A 1 B 0 B T X + P I n 1 0 X A 2 = 0 0 P F 1 MZ+D 0 ZA 2 = F F 2 with F low rank. We rewrite the matrix equation as ZA 1 2 +M 1 D 0 Z = F, F = M 1 FA 1 2, which is a Sylvester equation with a singular coefficient matrix. Z Z k = V k Z k W T k with Range(V k ),Range(W k ) appropriate approximation spaces of small dimensions 33

34 Large scale problem. Iterative method. 3/3 ZA 1 2 +M 1 D 0 Z = F l FT r Z Z k = V k Z k W T k Choice of V k,w k. A possible strategy: W k = EK k (A T 2, F r ), Extended Krylov subspace V k = K k (M 1 D 0, F l ) K k ((M 1 D 0 +σi) 1, F l ) Augmented Krylov subspace, σ R (see, e.g., Shank & Simoncini (2013)) Note: M has size (n 1 +m) (n 1 +m) (Compare with (n 1 n 2 +mn 2 ) (n 1 n 2 +mn 2 ) of the Kronecker form) 34

35 Numerical experiments A 1 X XA 2 +B T P = 0, BX = F 2 vs Az = f A 1 L 1 = u xx u yy A 2 L 1 = (e 10xy u x ) x (e 10xy u y ) y +10(x+y)u x [F 1 ;F 2 ] rank-1 matrix B = bidiag( 1,1) R (n 2 n 1 ) n 2, iterative: tol=10 6, σ = 10 2 Elapsed time n 1 n 2 size(a) Monolithic Iterative EK , e e-02 (4) , e e-02 (4) , e e-02 (4) , e e-01 (5) , e e-01 (5) ,962, e e+00 (5) 35

36 H BT B 0 Numerical experiments. 1D stochastic Stokes problem. 1/3 x = f 1, H = (ν 0 G 0 +ν 1 G 1 ) A x, B = G 0 B x p f 2 where ν = ν 0 +ν 1 ξ(ω) uncertain viscosity, ξ random variable Then A x XG 0 ν 0 +A x XG 1 ν 1 +B T xpg 0 = F 1, B x X = F 2 with G 0 = I. This corresponds to B T x ν 0A x B x 0 X + P ν 1A x X G 1 = 0 P F1 F2 that is MZ+D 0 ZG 1 = F 36

37 Numerical experiments. 1D stochastic Stokes problem. 2/3 MZ+D 0 ZG 1 = F vs Az = f n 1 n 2 size(a) Monolithic Iterative EK , (2) , (2) , (2) ν 0 = 1/10,ν 1 = 3ν 0 /10 Powell-Silvester (2012) n 2 could be much larger, n 2 = O(10 4 ) Memory requirements are very limited 37

38 Numerical experiments. 1D stochastic Stokes problem. 3/3 MZ+D 0 ZG 1 = F vs Az = f n 1 n 2 size(a) Monolithic Iterative EK , (2) , (2) , (2) ν 0 = 1/10,ν 1 = 3ν 0 /10 Powell-Silvester (2012) n 2 could be much larger, n 2 = O(10 3 ) Memory requirements are very limited, Z = Z 1 Z T 2 of very low rank 38

39 Numerical experiments. 2D stochastic Stokes problem H BT B 0 x = p f 1, f 2 H = blkdiag((ν 0 G 0 +ν 1 G 1 ) A x,(ν 0 G 0 +ν 1 G 1 ) A y ) B = [G 0 B x, G 0 B y ] MZ+D 0 ZG 1 = F vs Az = f n 1 n 2 size(a) Monolithic Iterative EK , (2) , (2) , (2) ν 0 = 1/10,ν 1 = 3ν 0 /10 Powell-Silvester (2012) 39

40 Not discussed but in this category Sylvester-like linear matrix equations AX +f(x)b = C typically (but not only!): f(x) = X, f(x) = X, or f(x) = X (Bevis, Braden, Byers, Chiang, De Terán, Dopico, Duan, Feng, Gonzalez, Guillery, Hall, Hartwig, Ikramov, Kressner, Montealegre, Reyes, Schröder, Vorntsov, Watkins, Wu,...) Linear systems with complex tensor structure Ax = b with A = k I n1 I nj 1 A j I nj+1 I nk. j=1 Dolgov, Grasedyck, Khoromskij, Kressner, Oseledets, Tobler, Tyrtyshnikov, and many more... 40

41 Not discussed but in this category Sylvester-like linear matrix equations AX +f(x)b = C typically (but not only!): f(x) = X, f(x) = X, or f(x) = X (Bevis, Braden, Byers, Chiang, De Terán, Dopico, Duan, Feng, Gonzalez, Guillery, Hall, Hartwig, Ikramov, Kressner, Montealegre, Reyes, Schröder, Vorntsov, Watkins, Wu,...) Linear systems with complex tensor structure Ax = b with A = k I n1 I nj 1 A j I nj+1 I nk. j=1 Dolgov, Grasedyck, Khoromskij, Kressner, Oseledets, Tobler, Tyrtyshnikov, and many more... 41

42 Conclusions Multiterm (Kron) linear equations is the new challenge Great advances in solving really large linear matrix equations Second order (matrix) challenges rely on strength and maturity of linear system solvers Low-rank tensor formats is the new generation of approximations Reference for linear matrix equations: V. Simoncini, Computational methods for linear matrix equations, SIAM Review, Sept