Matrix Equations and and Bivariate Function Approximation

Size: px

Start display at page:

Download "Matrix Equations and and Bivariate Function Approximation"

Robyn Harris
5 years ago
Views:

1 Matrix Equations and and Bivariate Function Approximation D. Kressner Joint work with L. Grasedyck, C. Tobler, N. Truhar. ETH Zurich, Seminar for Applied Mathematics Manchester,

2 Sylvester matrix equation linear matrix equation AX + XB = D (SYLV) with real n n coefficient matrices A, B, D. Applications: controllability/observability Gramians of linear systems (B = A T, D symmetric pos semidef) balanced truncation model reduction Newton methods for solving algebraic Riccati equation stability analysis 2D PDE with separable coefficients... D. Kressner p. 2

3 Outline Connection to bivariate functions X 1/(α + β) Approximation to X Approximation to 1/(α + β) Applications Singular value decay of X Convergence of Krylov subspace methods Convergence of extended Krylov subspace methods Extension to higher dimensions D. Kressner p. 3

4 Connection to bivariate functions D. Kressner p. 4

5 The diagonalizable case Assume A, B diagonalizable: P 1 AP = Λ A, Q 1 BQ = Λ B with Λ A = diag(α 1,...,α m ), Λ B = diag(β 1,...,β n ). Then AX + XA T = D Λ A X + XΛB = P 1 DQ =: D. with explicit solution x ij = d ij α i + β j D. Kressner p. 5

6 The diagonalizable case Assume A, B diagonalizable: P 1 AP = Λ A, Q 1 BQ = Λ B with Then Λ A = diag(α 1,...,α m ), Λ B = diag(β 1,...,β n ). AX + XA T = D Λ A X + XΛB = P 1 DQ =: D. with explicit solution d ij x ij = α i + β j X = D [ 1 ] α i + β j X is the Hadamard product between D and the bivariate function 1/(α + β) evaluated at Λ(A) Λ(B). D. Kressner p. 5

7 Separable approximations to bivariate functions Bivariate function of the form f(α)g(β) is called separable. 1/(α + β) is not separable but... D. Kressner p. 6

8 Separable approximations to bivariate functions Bivariate function of the form f(α)g(β) is called separable. 1/(α + β) is not separable but might be well approximated by short sums of separable functions: 1 r α + β s r(y, z) := f k (α)g k (β). k=1 D. Kressner p. 6

9 Separable approximations to bivariate functions Bivariate function of the form f(α)g(β) is called separable. 1/(α + β) is not separable but might be well approximated by short sums of separable functions: 1 r α + β s r(y, z) := f k (α)g k (β). k=1 Set X r = r f k (A)Dg k (B), then k=1 X r = D [ ] s r (α i,β j ) Hadamard product between D and s r (α,β) evaluated at Λ(A) Λ(B). D. Kressner p. 6

10 Approximation to X Approximation to 1/(α + β) AX + XB = D Let Λ(A) Ω A, Λ(B) Ω B and define r r X r = f k (A)Dg k (B), s r (α,β) = f k (α)g k (β), k=1 k=1 with functions f k : Ω A C, g k : Ω B C. X X r F κ(p)κ(q) 1 α + β s r(α,β) D F.,ΩA Ω B D. Kressner p. 7

11 Idea of proof If A = Λ A, B = Λ B are diagonal solution of Λ A X + XΛ B = D: d 11 d 12 α 1 +β 1 α 1 +β 2 X = d 21 d 22 α 2 +β 1 α 2 +β 2.. X X r = 1 P f α 1 +β k (α 1 )g k (β 1 ) 1 1 P f α 2 +β k (α 2 )g k (β 1 ) 1. 1 P f α 1 +β k (α 1 )g k (α 2 ) 2 1 P f α 2 +β k (α 2 )g k (α 2 ) 2. D D. Kressner p. 8

12 The non-diagonalizable case Consider contours Γ A,Γ B C enclosing Λ(A),Λ(B) with Γ A Γ B =. Then X = 1 1 4π 2 Γ B α + β (αi A) 1 D(βI B) 1 dαdβ Γ A solves AX + XB = D. D. Kressner p. 9

13 The non-diagonalizable case Consider contours Γ A,Γ B C enclosing Λ(A),Λ(B) with Γ A Γ B =. Then X = 1 1 4π 2 Γ B α + β (αi A) 1 D(βI B) 1 dαdβ Γ A solves AX + XB = D. Similarly, for r X r = f k (A)Dg k (A T ), s r (α,β) = we have k=1 X r = 1 4π 2 Γ A r f k (α)g k (β), k=1 Γ B s r (α,β)(αi A) 1 D(βI B) 1 dαdβ. D. Kressner p. 9

14 The non-diagonalizable case X X r F κ(ω A )κ(ω B ) 1 α + β s r(α,β) D F.,ΩA Ω B Constants : κ(ω A ) = 1 (αi A) 1 2 dα, 2π Γ A κ(ω B ) = 1 (βi B) 1 2 dβ. 2π Γ B See discussion in [Beattie, Embree, Sorensen, SIAM Review 05] on κ(ω A ) and κ(ω B ). D. Kressner p. 10

15 General notion of bivariate matrix functions? Consider two matrices A, B and contours Γ A,Γ B C enclosing Λ(A),Λ(B). For any function f(α,β) analytic on Γ A Γ B, one could define f(a, B)[D] = 1 4π 2 f(α,β)(αi A) 1 D(βI B) 1 dαdβ, Γ B Γ A as the evaluation of f at (A, B) applied to C. Such an abstract framework might be of use in other applications (Stein equations, Fréchet derivative of matrix functions). D. Kressner p. 11

16 Applications Singular value decay D. Kressner p. 12

17 Singular value decay Approximation results X X r F 1 α + β s r(α,β) D F.,ΩA Ω B If D has rank one X r = f k (A)Dg k (B) has rank r. σ r+1 (X) 1 α + β s r(α,β) D 2,ΩA Ω B No restriction on the choice of functions f k, g k. D. Kressner p. 13

18 Example: The symmetric case Assume A, B symmetric with eigenvalues contained in interval [λ min,λ max ] with λ min λ max < 0. [Braess 86], [Braess/Hackbusch 05]: Let s r (α,β) = r ω k exp(γ k α) exp(γ k β), k=1 ω k < 0,γ k > 0 s.t. [ ] 1/(α + β) s r (α,β) [λmin,λ max] 2 8 λ max exp rπ 2 log(8 λ. min λ max ) D. Kressner p. 14

19 Example: The symmetric case If D has rank one: [ σ r+1 (X) 8 A 1 rπ 2 ] 2 exp D 2. log(8κ(a)) This matches an existing bound in [Sabino 06] for Lyapunov equations. Results for complex Λ(A), Λ(B) can be obtained by sinc interpolation [Grasedyck/Hackbusch/Khoromskij 02, Grasedyck/K. 09]. Isolated eigenvalues close to zero can be matched by exact polynomial interpolation. D. Kressner p. 15

20 Lyapunov equation: Singular value decay Other existing decay bounds: [Penzl 00]: r 1 σ r+1 (X) κ (2j+1)/(2k) 1 X 2 κ (2j+1)/(2k) + 1 j=0 [Antoulas/Sorensen/Zhou 02]: r 1 λ r λ j σ r (X) λ r + λ j j=1 [Grasedyck/Hackbusch/Khoromskij 02]: σ r+1 (X) X 2 2 ( ( = exp O 2 exp( r).. r log κ(a) )). D. Kressner p. 16

21 Example: A R second-difference operator and D = bb T with b = [1,...,1] T GHK bound Penzl s bound ASZ bound exponential bound singular values D. Kressner p. 17

22 Applications Subspace projection methods D. Kressner p. 18

23 Basic idea of subspace projection methods Approximate solution to AX + XA T = bb T using a subspace U R n that contains Krylov subspace for A: K r (A, b) = span{b, Ab, A 2 b,...,a r 1 b} D. Kressner p. 19

24 Basic idea of subspace projection methods Approximate solution to AX + XA T = bb T using a subspace U R n that contains Krylov subspace for A: K r (A, b) = span{b, Ab, A 2 b,...,a r 1 b} Krylov subspace for A 1 : K r (A 1, b) = span{b, A 1 b,...,a r+1 b} D. Kressner p. 19

25 Basic idea of subspace projection methods Approximate solution to AX + XA T = bb T using a subspace U R n that contains Krylov subspace for A: K r (A, b) = span{b, Ab, A 2 b,...,a r 1 b} Krylov subspace for A 1 : K r (A 1, b) = span{b, A 1 b,...,a r+1 b} Krylov subspaces for (A σi) 1 : K r ((A σi) 1, b) = span{b,(a σi) 1 b,...,(a σi) r+1 b} D. Kressner p. 19

26 Convergence of Arnoldi for Lyapunov 2D instationary heat equation on the unit square with homogeneous kind boundary conditions; finite difference discretization Approximation error Arnoldi steps Linear convergence observed. D. Kressner p. 20

27 Convergence does not capture singular value decay Approximation error from Arnoldi method Singular values of X Example: svd(x) tells there is rank 12 approximation with error 10 8 ; Arnoldi yields only rank 84 approximation with error D. Kressner p. 21

28 1 Arnoldi 10 0 Extended Arnoldi Rational Arnoldi (6 shifts) Rational Arnoldi (13 shifts) Convergence for Steel example (Oberwolfach collection).

29 Galerkin condition From now on: A is symmetric and Λ(A) [λ min,λ max ] with λ min λ max < 0. Approximation X r satisfies Galerkin condition Equivalent to AX r + X r A + bb T U U. X X r A = min Z U U X Z A, in the weighted Frobenius norm C A := vec(c) (I A+A I). D. Kressner p. 23

30 Convergence of Arnoldi method If U = K r (A, b), every Z U U takes the form Hence, Z = r 1 i,j=0 γ ij A i bb T A j, γ ij R. X X r A A 2 min 1 r 1 γ ij R α + β γ ij α i β j [λmin,λ max] 2 b 2 2. i,j=1 D. Kressner p. 24

31 Convergence of Arnoldi method Error for separable polynomial approximation of 1/(α + β): 1 κ + 1 λ max ( κ 1) r κ ( κ + 1) r with κ = ( λ min + λ max )/(2 λ min ). Proof based on careful analysis of Chebyshev approximation to exponentials. D. Kressner p. 25

32 Convergence of Arnoldi method Error for separable polynomial approximation of 1/(α + β): 1 κ + 1 λ max ( κ 1) r κ ( κ + 1) r with κ = ( λ min + λ max )/(2 λ min ). Proof based on careful analysis of Chebyshev approximation to exponentials. Convergence bound for Arnoldi method: X X r A A 2 A 1 2 b 2 2 κ + 1 κ ( κ 1) r ( κ + 1) r, with κ κ(a)/2. Matches a bound in [Simoncini/Druskin 09]. D. Kressner p. 25

33 Convergence of Arnoldi method Observed convergence and convergence bound. D. Kressner p. 26

34 Convergence of Extended Arnoldi method Observed convergence for U = span ( K r (A, b) K r (A 1, b) ). D. Kressner p. 27

35 Convergence of Extended Arnoldi method U = span ( K r (A, b) K r (A 1, b) ) = {p(a)b : p L r } where L r denotes the set of Laurent polynomials γ r α r + + γ 1 α 1 + γ 0 + γ 1 α + + γ r 1 α r 1. X X r A A 2 min 1 r 1 γ ij R α + β γ ij α i β j b 2 2. i,j= r D. Kressner p. 28

36 Convergence of extended Arnoldi method Error for separable Laurent polynomial approximation of 1/(α + β): 1 λ max κ1/4 + 1 κ 1/4 (κ1/4 1) r (κ 1/4 + 1) r with κ = ( λ min + λ max )/(2 λ max ). Proof based on polynomial approximation: If p is polynomial, p(α + γ/α) is Laurent polynomial. D. Kressner p. 29

37 Convergence of extended Arnoldi method Error for separable Laurent polynomial approximation of 1/(α + β): 1 λ max κ1/4 + 1 κ 1/4 (κ1/4 1) r (κ 1/4 + 1) r with κ = ( λ min + λ max )/(2 λ max ). Proof based on polynomial approximation: If p is polynomial, p(α + γ/α) is Laurent polynomial. Convergence bound for extended Arnoldi method: X X r A A 2 A 1 2 b 2 2 κ1/4 + 1 κ 1/4 with κ κ(a)/2 [K./Tobler 09]. (κ1/4 1) r (κ 1/4 + 1) r, D. Kressner p. 29

38 High-dimensional Extensions D. Kressner p. 30

39 A toy PDE u = f in Ω, u Ω = 0, Tensorize 1D FE bases V = {v 1 (x 1 ),..., v m (x 1 )} { W = {w 1 (x 2 ),..., w n (x 2 )} V W = αij } v i (x 1 )w j (x 2 ) Galerkin discretization (K V M W + M V K W )u = f, with 1D mass/stiffness matrices M V, M W, K V, K W ; denotes Kronecker product. D. Kressner p. 31

40 reshape u u = u 1 u 2 MATLAB: U = reshape(u,m,n);.,u j R m U = [u 1, u 2,...] (K V M W )u K V UM W. D. Kressner p. 32

41 A toy PDE u = f in Ω, u Ω = 0, Tensorize 1D FE bases V = {v 1 (x 1 ),..., v m (x 1 )} { W = {w 1 (x 2 ),..., w n (x 2 )} V W = αij } v i (x 1 )w j (x 2 ) Galerkin discretization K V UM W + M V UK W = F, with 1D mass/stiffness matrices M V, M W, K V, K W ; u = vec(u), f = vec(f). D. Kressner p. 33

42 A toy PDE u = f in Ω, u Ω = 0, Tensorize 1D FE bases V = {v 1 (x 1 ),..., v m (x 1 )} { W = {w 1 (x 2 ),..., w n (x 2 )} V W = αij } v i (x 1 )w j (x 2 ) Galerkin discretization ( 1/2 M V K V M 1/2 ) ( 1/2 V X + X M W K W M 1/2 ) 1/2 W = M V FM 1/2 W, with 1D mass/stiffness matrices M V, M W, K V, K W ; X = M 1/2 V UM1/2 W, f = vec(f). D. Kressner p. 34

43 The toy PDE in 3D u = f in Ω, u Ω = 0, Tensorize 1D FE bases V = {v 1 (x 1 ),..., v m (x 1 )} W = {w 1 (x 2 ),..., w n (x 2 )} Z = {z 1 (x 3 ),..., z n (x 3 )} V W Z = { αijk v i (x 1 )w j (x 2 )z k (x 3 )} Galerkin discretization (K V M W M Z + M V K W M Z + M V M W K Z )u = f, with 1D mass/stiffness matrices M V, M W, M Z, K V, K W, K Z. D. Kressner p. 35

44 Higher-dimensional matrix equations Kroneckerized Sylvester equation (A 1 I + I A 2 )x = b. Generalization to dimension d = 3: (A 1 I I + I A 2 I + I I A 3 )x = b. Generalization to arbitrary dimension: Ax = b, A = d I I A j I I. j=1 D. Kressner p. 36

45 Properties Ax = b, A = d I I A j I I. j=1 If A j R n n then A R nd n d. Curse of dimension! Λ(A) = {λ (1) + + λ (d) : λ (j) Λ(A j )}. If κ(a j ) κ then κ(a) = κ. If b = b 1 b 2 b d (rank-1 tensor) and all A j spd solution x can be well approximated by short sum of rank-1 tensors [Grasedyck 04, K./Tobler 09]. D. Kressner p. 37

46 Tensor Krylov subspace method Arnoldi method for Lyapunov equations extends to d > 2 for b = b 1 b d. Works with K r (A j, b j ) (Krylov subspace in each individual direction). Approximates x by x r satisfying Galerkin condition If all A j spd: Ax r b span(k r (A 1, b 1 ) K r (A d, b d )). x x r 2 ( κ 1 κ + 1 ) r, κ κ(a) d. D. Kressner p. 38

47 CG vs. Tensor Krylov Comp. Complexity Storage requ. CG O(r 2 n d ) O(rn d ) Tensor Krylov O(r 2 dn) O(rdn) D. Kressner p. 39

48 10 2 dim = 5 dim = dim = Example: Laplace on [0, 1] d, n = 1024, n Tensor Krylov requires 300 iterations and 1h to attain residual 10 6 for d = 50.

49 Conclusions Singular value decay/projection methods for Sylvester equations bivariate function approximation. Low tensor rank approximation/projection methods for linear systems with tensor product structure multivariate function approximation. See for further details. D. Kressner p. 41

50 Workshop on Matrix Equations Fr, 9th October 2009 TU Braunschweig, Germany Organizers: Peter Benner, TU Chemnitz Heike Faßbender, TU Braunschweig Lars Grasedyck, MPI Leipzig Daniel Kressner, ETH Zurich Contact:

Krylov subspace methods for linear systems with tensor product structure

Krylov subspace methods for linear systems with tensor product structure Christine Tobler Seminar for Applied Mathematics, ETH Zürich 19. August 2009 Outline 1 Introduction 2 Basic Algorithm 3 Convergence