On the properties of Krylov subspaces in finite precision CG computations

Transcription

1 On the properties of Krylov subspaces in finite precision CG computations Tomáš Gergelits, Zdeněk Strakoš Department of Numerical Mathematics, Faculty of Mathematics and Physics, Charles University in Prague 4th IMA Conference, Birmingham 3rd September, th IMA Conference T. Gergelits Krylov subspaces in FP CG /12

2 Content of the talk 1 The essence of the CG method 2 Krylov subspaces in practical computations 3 Idea of shift 4 Comparison of trajectories of approximation vectors 5 Comparison of generated Krylov subspaces 6 Concluding remarks 2 / 12

3 The essence of the CG method Consider preconditioned system Ax = b, A F N N HPD, b F N, F is R or C CG is the projection method which minimizes the energy norm of the error x k x 0 + K k (A, r 0 ), r k K k (A, r 0 ), k = 1, 2,... K k (A, r 0 ) = span{r 0, Ar 0, A 2 r 0,..., A k 1 r 0 } x x k A = min { x y A : y x 0 + K k (A, r 0 )}. 3 / 12

4 Krylov subspaces in practical computations Krylov subspace K k (B, v) = span{v, Bv,..., B k 1 v} is built up by powering the matrix. 4 / 12

5 Krylov subspaces in practical computations Krylov subspace K k (B, v) = span{v, Bv,..., B k 1 v} is built up by powering the matrix. Important question arising in numerical computations What is the difference between K k (B, v) and K k (B, v)? 4 / 12

6 Krylov subspaces in practical computations Krylov subspace K k (B, v) = span{v, Bv,..., B k 1 v} is built up by powering the matrix. Important question arising in numerical computations What is the difference between K k (B, v) and K k (B, v)? Related question of sensitivity of Krylov subspaces K k (B + B, v + δv). 4 / 12

7 Krylov subspaces in practical computations Krylov subspace K k (B, v) = span{v, Bv,..., B k 1 v} is built up by powering the matrix. Important question arising in numerical computations What is the difference between K k (B, v) and K k (B, v)? Related question of sensitivity of Krylov subspaces K k (B + B, v + δv). Perturbation analysis, condition number of Krylov subspaces. [Carproux, Godunov, Kuznetsov (1997); Paige, Van Dooren (1998)] 4 / 12

8 Krylov subspaces in practical computations Krylov subspace K k (B, v) = span{v, Bv,..., B k 1 v} is built up by powering the matrix. Important question arising in numerical computations What is the difference between K k (B, v) and K k (B, v)? Related question of sensitivity of Krylov subspaces K k (B + B, v + δv). Perturbation analysis, condition number of Krylov subspaces. [Carproux, Godunov, Kuznetsov (1997); Paige, Van Dooren (1998)] Short recurrences = significant delay of convergence. 4 / 12

9 CG in finite precision computations Short recurrences 5 / 12

10 CG in finite precision computations Short recurrences = loss of orthogonality 5 / 12

11 CG in finite precision computations Short recurrences = loss of orthogonality = delay of convergence & rank deficiency 10 0 exact arithmetic finite precision arithmetic loss of orthogonality relative A norm of the error delay of convergence iteration number 5 / 12

12 Idea of shift We relate: k-th iteration of FP CG l-th iteration of exact CG k l delay of convergence k l rank-deficiency of computed Krylov subspace We want to study: x x k A x x l A x k x l K k (A, r 0 ) K l (A, r 0 ) 6 / 12

13 Comparison of trajectory of approximation vectors 10 0 energy norm x x l A x x l A iteration number 7 / 12

14 Comparison of trajectory of approximation vectors 10 0 energy norm x x l A x x k A (6) 12 (12) 18 (23) 25 (50) l(k) 7 / 12

15 Comparison of trajectory of approximation vectors 10 0 energy norm x x l A x x k A x k x l A (6) 12 (12) 18 (23) 25 (50) l(k) 7 / 12

16 Comparison of trajectory of approximation vectors energy norm x x l A x x k A x k x l A xk xl A x xl A Observation x k x l A x x l A (6) 12 (12) 18 (23) 25 (50) l(k) Trajectories of approximation vectors are very similar in space F N. 7 / 12

17 Comparison of trajectory of approximation vectors Ax = b F N Ax = b F N x kxl x delay at the k-th step x k x k x exact computation finite precision computation x 0 exact computation finite precision computation Trajectory of approximations x k generated by FP CG computations follows closely the trajectory of the exact CG approximations x l. 8 / 12

18 Comparison of Krylov subspaces Canonical angles and vectors ϑ j = min p F j p =1 min arccos ( p q ) arccos ( p j q j ), q G j q =1 j = 1, 2,..., l where F j F {p 1,..., p j 1 }, G j G {q 1,..., q j 1 }, F = K k (A, r 0 ), G = K l (A, r 0 ). angles - in degrees ϑ l ϑ l 1 ϑ l 2 ϑ l 3 Comparison of principle angles of subspaces K k and K l (4) 8 (8) 12 (12) 16 (19) 20 (29) 25 (54) l(k) 9 / 12

19 Things are not so nice (Data: bus494 from MatrixMarket) 10 0 energy norm x x l A x x k A x k x l A xk xl A x xl A angles - in degrees (187) 185 (537) 277 (1083) 370 (1776) l(k) Comparison of principle angles of subspaces K k and K l ϑ l ϑ l 1 ϑ l 2 ϑ l (155) 164 (446) 247 (870) 329 (1475) 411 (1888) 494 (2042) l(k) 10 / 12

20 Influence of clustered eigenvalues Cluster of 3 largest eigenvalues with width. = 10 3 = 10 8 = angles - in degrees angles - in degrees angles - in degrees ϑl ϑl 1 ϑl 2 ϑl 3 Comparison of principle angles of subspaces Kk and Kl (4) 9 (9) 14 (14) 18 (22) 23 (41) 28 (47) l(k) ϑl ϑl 1 ϑl 2 ϑl 3 Comparison of principle angles of subspaces Kk and Kl (4) 9 (9) 14 (14) 18 (21) 23 (39) 28 (45) l(k) ϑl ϑl 1 ϑl 2 ϑl 3 Comparison of principle angles of subspaces Kk and Kl (4) 9 (9) 14 (14) 18 (21) 23 (38) 28 (43) l(k) 11 / 12

21 Summary and outlook The trajectories of computed approximations are enclosed in a shrinking cone. Observed stability (or inertia?) of computed Krylov subspaces represents phenomenon which needs to be further studied. How to determine pairs (l, k). Effect of clustered eigenvalues. Theoretical proofs, relationship to the structure of invariant subspaces. Principle difference between long and short recurrences. 12 / 12

22 Summary and outlook The trajectories of computed approximations are enclosed in a shrinking cone. Observed stability (or inertia?) of computed Krylov subspaces represents phenomenon which needs to be further studied. How to determine pairs (l, k). Effect of clustered eigenvalues. Theoretical proofs, relationship to the structure of invariant subspaces. Principle difference between long and short recurrences. Acknowledgement This work has been supported by the ERC-CZ project LL1202, by the GACR grant 201/ S and by the GAUK grant / 12