Numerical behavior of inexact linear solvers
|
|
- Cora Hood
- 6 years ago
- Views:
Transcription
1 Numerical behavior of inexact linear solvers Miro Rozložník joint results with Zhong-zhi Bai and Pavel Jiránek Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic The fourth international conference on numerical algebra and scientific computing (NASC 2012), Dalian University of Technology, Dalian, China, October 20-24, 2012
2 Iterative solution of linear systems Iterative methods in exact arithmetic generate approximate solutions to the solution of Ax = b x 0, x 1,..., x n x with residual vectors r 0 = b Ax 0,..., r n = b Ax n 0 Iterative methods in finite precision arithmetic compute approximations x 0, x 1,..., x n and updated residual vectors r 0, r 1,..., r n which are usually close to (but different from) the true residuals b A x n
3 Two main questions and two main effects How good is the computed approximate solution x n? How many (extra) steps do we need to reach the same accuracy as one can get in the exact method? How well the computed vector r n approximates the (true) residual b A x n? Is there a limitation on the accuracy of the computed approximate solution? Two main effects of rounding errors: DELAY OF CONVERGENCE AND LIMIT ON THE FINAL (MAXIMUM ATTAINABLE) ACCURACY
4 Delay of convergence and maximum attainable accuracy
5 The concept of backward stability A backward stable algorithm eventually computes the exact answer to a nearby problem, i.e. the approximate solution x n satisfying (A + A n ) x n = b + b n A n / A O(u), b n / b O(u) The normwise backward error associated with the approximate solution x n satisfies b A x n b + A x O(u) n A forward stable algorithm eventually computes the approximate solution x n with the error that satisfies x n x O(u) A 1 A x Prager, Oettli, 1964; Rigal, Gaches, 1967 see also Higham, 2nd ed. 2002; Stewart, Sun, 1990; Meurant 1999
6 The level of maximum attainable accuracy We are looking for the difference between the updated r n and true residual b A x n (divided by b + A x n ) b A x n r n b + A x n? r n 0 = lim n b A x n b + A x n? In the optimal case the bound is of O(u); then we have a backward stable solution. The backward stability implies the forward stability. Higham 2002, Higham, Knight 1993, Greenbaum, R, Strakoš, 1997
7 Stationary iterative methods Ax = b, A = M N A: Mx k+1 = N x k + b B: x k+1 = x k + M 1 (b Ax k ) Inexact solution of systems with M: every computed solution y of My = z is interpreted as an exact solution of a system with perturbed data and relative perturbation bounded by parameter τ such that (M + M)y = z, M τ M, τk(m) 1
8 Accuracy of the computed approximate solution A: Mx k+1 = N x k + b ˆx k+1 x x τ M 1 ( M + N ) b Aˆx k+1 M + N τ b + A ˆx k+1 A B: x k+1 = x k + M 1 (b Ax k ) ˆx k+1 x x O(u) M 1 A ( 1 + max ) i=0,...,k{ ˆx i } x ( 1 + max i=0,...,k{ ˆx i } x ( 1 + max ) i=0,...,k{ ˆx i } x ( b Aˆx k+1 b + A ˆx k+1 O(u) 1 + max ) i=0,...,k{ ˆx i } x ) Higham, Knight 1993, Bai, R, 2012
9 Numerical experiments: small model example A = tridiag(1, 4, 1) R , b = A ones(100, 1), κ(a) = A A 1 = A = M N, M = D L, N = U
10 τ = 10 2 relative error norms τ = 10 6 τ = τ = O(u) iteration number k
11 τ = 10 2 normwise backward error τ = 10 6 τ = τ = O(u) iteration number k
12 relative error norms τ = O(u), τ=, τ=10 6, τ= iteration number k
13 normwise backward errors τ = O(u), τ=, τ=10 6, τ= iteration number k
14 Two-step splitting iteration methods M 1x k+1/2 = N 1x k + b, A = M 1 N 1 M 2x k+1 = N 2x k+1/2 + b, A = M 2 N 2 Numerous solution schemes: Hermitian/skew-Hermitian (HSS) splitting, modified Hermitian/skew-Hermitian (MHSS) splitting, normal Hermitian/skew-Hermitian (NSS) splitting, preconditioned variant of modified Hermitian/skew-Hermitian (PMHSS) splitting and other splittings,... Bai, Golub, Ng 2003, 2007, 2008; Bai 2009 Bai, Benzi, Chen 2010, 2011; Bai, Benzi, Chen, Wang 2012 ˆx k+1 x x [ τ 1 M 1 2 N 2 M 1 1 ( M 1 + N 1 ) + τ 2 M 1 2 ( M 2 + N ] 2 ) ( 1 + max ) i=0,1/2,...,k+1/2{ ˆx i } x
15 Two-step splitting iteration methods x k+1/2 = x k + M 1 1 (b Ax k ) x k+1 = x k+1/2 + M 1 2 (b Ax k+1/2 ) x k+1 = x k + (M M 1 2 M 1 2 AM 1 1 )(b Ax k ) = x k + (I + M 1 2 N 1)M 1 1 (b Ax k ) = x k + M 1 2 (I + N 2M 1 1 )(b Ax k ) ˆx k+1 x x ( O(u) M 1 2 (I + N 2M 1 1 ) A 1 + max ) i=0,...,k{ ˆx i } x
16 Numerical experiments: small model example A = tridiag(2, 4, 1) R , b = A ones(100, 1), κ(a) = A A 1 = A = H + S, H = 1 2 (A + AT ), S = 1 2 (A AT ) H = tridiag( 3 2, 4, 3 2 ), S = tridiag( 1 2, 0, 1 2 )
17 10 0 relative error norms τ 1 = 10 2, τ 2 = τ 1 = 10 6, τ 2 = τ 1 =, τ 2 = τ 1 = O(u), τ 2 = τ 1 =O(u),, 10 6, 10 2, τ 2 = iteration number k
18 relative error norms τ 1 =, τ 2 =10 2 τ 1 =, τ 2 =10 6 τ 1 =, τ 2 = τ 1 =, τ 2 =O(u) τ 1 =, τ 2 =O(u),, 10 6, iteration number k
19 Saddle point problems We consider a saddle point problem with the symmetric 2 2 block form ( ) ( ) ( ) A B x f B T =. 0 y 0 A is a square n n nonsingular (symmetric positive definite) matrix, B is a rectangular n m matrix of (full column) rank m. Applications: mixed finite element approximations, weighted least squares, constrained optimization, computational fluid dynamics, electromagnetism etc. [Benzi, Golub and Liesen, 2005], [Elman, Silvester, Wathen, 2005]. For the updated list of applications leading to saddle point problems contact [Benzi].
20 Iterative solution of saddle point problems Numerous solution schemes: inexact Uzawa algorithms, inexact null-space methods, inner-outer iteration methods, two-stage iteration processes, multilevel or multigrid methods, domain decomposition methods Numerous preconditioning techniques and schemes: block diagonal preconditioners, block triangular preconditioners, constraint preconditioning, Hermitian/skew-Hermitian preconditioning and other splittings, combination preconditioning Numerous iterative solvers: conjugate gradient (CG) method, MINRES, GMRES, flexible GMRES, GCR, BiCG, BiCGSTAB,...
21 Schur complement reduction method Compute y as a solution of the Schur complement system B T A 1 By = B T A 1 f, compute x as a solution of Ax = f By. Segregated vs. coupled approach: x k and y k approximate solutions to x and y, respectively. Inexact solution of systems with A: every computed solution û of Au = b is interpreted as an exact solution of a perturbed system (A + A)û = b + b, A τ A, b τ b, τκ(a) 1.
22 Iterative solution of the Schur complement system choose y 0, solve Ax 0 = f By 0 compute α k and p (y) k y k+1 = y k + α k p (y) k solve Ap (x) k = Bp (y) k back-substitution: A: x k+1 = x k + α k p (x) k, B: solve Ax k+1 = f By k+1, C: solve Au k = f Ax k By k+1, x k+1 = x k + u k. r (y) k+1 = r(y) k α k B T p (x) k inner iteration outer iteration
23 Accuracy in the saddle point system f Ax k By k O(α1)κ(A) 1 τκ(a) ( f + B Y k), B T x k r (y) k O(α2)κ(A) 1 τκ(a) A 1 B ( f + B Y k ), Y k max{ y i i = 0, 1,..., k}. Back-substitution scheme α 1 α 2 A: Generic update x k+1 = x k + α k p (x) τ u k B: Direct substitution τ τ x k+1 = A 1 (f By k+1 ) C: Corrected dir. subst. u τ x k+1 = x k + A 1 (f Ax k By k+1 ) } additional system with A B T A 1 f + B T A 1 By k = B T x k B T A 1 (f Ax k By k )
24 Numerical experiments: a small model example A = tridiag(1, 4, 1) R , B = rand(100, 20), f = rand(100, 1), κ(a) = A A 1 = , κ(b) = B B = [R, Simoncini, 2002]
25 Generic update: x k+1 = x k + α k p (x) k 10 0 τ = residual norm f Ax k By k τ = 10 6 τ = τ = O(u) relative residual norms B T x k / B T x 0, r k (y) / r0 (y) τ = O(u), τ =, τ=10 6, τ= iteration number k iteration number k
26 Direct substitution: x k+1 = A 1 (f By k+1 ) 10 0 τ = τ = 10 2 residual norm f Ax k By k τ = 10 6 τ = τ = O(u) relative residual norms B T x k / B T x 0, r k (y) / r0 (y) τ = 10 6 τ = τ = O(u) iteration number k iteration number k
27 Corrected direct substitution: x k+1 = x k + A 1 (f Ax k By k+1 ) τ = 10 2 residual norm f Ax k By k τ = O(u), τ =, τ=10 6, τ=10 2 relative residual norms B T x k / B T x 0, r k (y) / r0 (y) τ = 10 6 τ = τ = O(u) iteration number k iteration number k
28 Null-space projection method compute x N(B T ) as a solution of the projected system (I Π)A(I Π)x = (I Π)f, compute y as a solution of the least squares problem Π = B(B T B) 1 B T By f Ax, is the orthogonal projector onto R(B). Schemes with the inexact solution of least squares with B. Every computed approximate solution v of a least squares problem Bv c is interpreted as an exact solution of a perturbed least squares (B + B) v c + c, B τ B, c τ c, τκ(b) 1.
29 Null-space projection method choose x 0, solve By 0 f Ax 0 compute α k and p (x) k N(B T ) x k+1 = x k + α k p (x) k solve Bp (y) k r (x) k α k Ap (x) k back-substitution: A: y k+1 = y k + p (y) k, B: solve By k+1 f Ax k+1, C: solve Bv k f Ax k+1 By k, y k+1 = y k + v k. r (x) k+1 = r(x) k α k Ap (x) k Bp (y) k inner iteration outer iteration
30 Accuracy in the saddle point system f Ax k By k r (x) k O(α3)κ(B) 1 τκ(b) ( f + A X k), B T x k O(τ)κ(B) 1 τκ(b) B X k, X k max{ x i i = 0, 1,..., k}. Back-substitution scheme α 3 A: Generic update y k+1 = y k + p (y) u k B: Direct substitution τ y k+1 = B (f Ax k+1 ) C: Corrected dir. subst. u y k+1 = y k + B (f Ax k+1 By k ) } additional least square with B
31 Generic update: y k+1 = y k + p (y) k τ = 10 2 relative residual norms f Ax k By k / f Ax 0 By 0, r (x) k / r(x) τ = O(u), τ =, τ=10 6, τ=10 2 residual norm B T x k τ = 10 6 τ = τ = O(u) iteration number iteration number
32 Direct substitution: y k+1 = B (f Ax k+1 ) 10 0 τ = τ = 10 2 relative residual norms f Ax k By k / f Ax 0 By 0, r (x) k / r(x) τ = 10 6 τ = τ = O(u) residual norm B T x k τ = 10 6 τ = τ = O(u) iteration number iteration number
33 Corrected direct substitution: y k+1 = y k + B (f Ax k+1 By k ) τ = 10 2 relative residual norms f Ax k By k / f Ax 0 By 0, r (x) k / r(x) τ = O(u), τ =, τ=10 6, τ=10 2 residual norm B T x k τ = 10 6 τ = τ = O(u) iteration number iteration number
34 The maximum attainable accuracy of saddle point solvers The accuracy measured by the residuals of the saddle point problem depends on the choice of the back-substitution scheme [Jiránek, R, 2008]. The schemes with (generic or corrected substitution) updates deliver approximate solutions which satisfy either the first or second block equation to working accuracy. Care must be taken when solving nonsymmetric systems [Jiránek, R, 2008], all bounds of the limiting accuracy depend on the maximum norm of computed iterates, cf. [Greenbaum 1994,1997], [Sleijpen, et al. 1994]. relative residual norms B T A 1 f B T A 1 By / B T A 1 f and r (y) k k / BT A 1 f iteration number k
35 Conclusions new value = old value + small correction Stationary iterative methods for Ax = b and their maximum attainable accuracy [Higham and Knight, 1993]: assuming splitting A = M N and inexact solution of systems with M, use x new = x old + M 1 (b Ax old ) rather than x new = M 1 (Nx old + b), [Higham, 2002; Bai, R]. Two-step splitting iteration framework: A = M 1 N 1 = M 2 N 2 assuming inexact solution of systems with M 1 and M 2, reformulation of M 1x 1/2 = N 1x old + b, M 2x new = N 2x 1/2 + b, Hermitian/skew-Hermitian splitting (HSS) iteration [Bai, Golub and Ng 2003; Bai, R]. Saddle point problems and inexact linear solvers: Schur complement and null-space approach [Jiránek, R 2008] Preconditioners for saddle point problems: SIMPLE and SIMPLE(R) type algorithms [Vuik and Saghir, 2002] and constraint preconditioners [R, Simoncini, 2002]. Fixed-precision iterative refinement for improving the computed solution x old to a system Ax = b: solving update equations Az corr = r that have residual r = b Ay old as a right-hand side to obtain x new = x old + z corr, see [Wilkinson, 1963], [Higham, 2002].
36 Thank you for your attention. miro Zhong-Zhi Bai and M. Rozložník, On the behavior of two-step splitting iteration methods, in preparation. P. Jiránek and M. Rozložník. Maximum attainable accuracy of inexact saddle point solvers. SIAM J. Matrix Anal. Appl., 29(4): , P. Jiránek and M. Rozložník. Limiting accuracy of segregated solution methods for nonsymmetric saddle point problems. J. Comput. Appl. Math. 215 (2008), pp M. Rozložník and V. Simoncini, Krylov subspace methods for saddle point problems with indefinite preconditioning, SIAM J. Matrix Anal. Appl., 24 (2002), pp
On the accuracy of saddle point solvers
On the accuracy of saddle point solvers Miro Rozložník joint results with Valeria Simoncini and Pavel Jiránek Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic Seminar at
More informationA stable variant of Simpler GMRES and GCR
A stable variant of Simpler GMRES and GCR Miroslav Rozložník joint work with Pavel Jiránek and Martin H. Gutknecht Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic miro@cs.cas.cz,
More informationHOW TO MAKE SIMPLER GMRES AND GCR MORE STABLE
HOW TO MAKE SIMPLER GMRES AND GCR MORE STABLE PAVEL JIRÁNEK, MIROSLAV ROZLOŽNÍK, AND MARTIN H. GUTKNECHT Abstract. In this paper we analyze the numerical behavior of several minimum residual methods, which
More informationc 2008 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 30, No. 4, pp. 1483 1499 c 2008 Society for Industrial and Applied Mathematics HOW TO MAKE SIMPLER GMRES AND GCR MORE STABLE PAVEL JIRÁNEK, MIROSLAV ROZLOŽNÍK, AND MARTIN
More informationInexactness and flexibility in linear Krylov solvers
Inexactness and flexibility in linear Krylov solvers Luc Giraud ENSEEIHT (N7) - IRIT, Toulouse Matrix Analysis and Applications CIRM Luminy - October 15-19, 2007 in honor of Gérard Meurant for his 60 th
More informationJournal of Computational and Applied Mathematics. Optimization of the parameterized Uzawa preconditioners for saddle point matrices
Journal of Computational Applied Mathematics 6 (009) 136 154 Contents lists available at ScienceDirect Journal of Computational Applied Mathematics journal homepage: wwwelseviercom/locate/cam Optimization
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More informationEfficient Solvers for the Navier Stokes Equations in Rotation Form
Efficient Solvers for the Navier Stokes Equations in Rotation Form Computer Research Institute Seminar Purdue University March 4, 2005 Michele Benzi Emory University Atlanta, GA Thanks to: NSF (MPS/Computational
More informationMathematics and Computer Science
Technical Report TR-2010-026 On Preconditioned MHSS Iteration Methods for Complex Symmetric Linear Systems by Zhong-Zhi Bai, Michele Benzi, Fang Chen Mathematics and Computer Science EMORY UNIVERSITY On
More informationStructured Preconditioners for Saddle Point Problems
Structured Preconditioners for Saddle Point Problems V. Simoncini Dipartimento di Matematica Università di Bologna valeria@dm.unibo.it. p.1/16 Collaborators on this project Mario Arioli, RAL, UK Michele
More informationPreconditioning for Nonsymmetry and Time-dependence
Preconditioning for Nonsymmetry and Time-dependence Andy Wathen Oxford University, UK joint work with Jen Pestana and Elle McDonald Jeju, Korea, 2015 p.1/24 Iterative methods For self-adjoint problems/symmetric
More informationChebyshev semi-iteration in Preconditioning
Report no. 08/14 Chebyshev semi-iteration in Preconditioning Andrew J. Wathen Oxford University Computing Laboratory Tyrone Rees Oxford University Computing Laboratory Dedicated to Victor Pereyra on his
More informationStructured Preconditioners for Saddle Point Problems
Structured Preconditioners for Saddle Point Problems V. Simoncini Dipartimento di Matematica Università di Bologna valeria@dm.unibo.it p. 1 Collaborators on this project Mario Arioli, RAL, UK Michele Benzi,
More informationPreconditioners for the incompressible Navier Stokes equations
Preconditioners for the incompressible Navier Stokes equations C. Vuik M. ur Rehman A. Segal Delft Institute of Applied Mathematics, TU Delft, The Netherlands SIAM Conference on Computational Science and
More informationPrinciples and Analysis of Krylov Subspace Methods
Principles and Analysis of Krylov Subspace Methods Zdeněk Strakoš Institute of Computer Science, Academy of Sciences, Prague www.cs.cas.cz/~strakos Ostrava, February 2005 1 With special thanks to C.C.
More informationFast Iterative Solution of Saddle Point Problems
Michele Benzi Department of Mathematics and Computer Science Emory University Atlanta, GA Acknowledgments NSF (Computational Mathematics) Maxim Olshanskii (Mech-Math, Moscow State U.) Zhen Wang (PhD student,
More informationThe Nullspace free eigenvalue problem and the inexact Shift and invert Lanczos method. V. Simoncini. Dipartimento di Matematica, Università di Bologna
The Nullspace free eigenvalue problem and the inexact Shift and invert Lanczos method V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1 The
More informationSplitting Iteration Methods for Positive Definite Linear Systems
Splitting Iteration Methods for Positive Definite Linear Systems Zhong-Zhi Bai a State Key Lab. of Sci./Engrg. Computing Inst. of Comput. Math. & Sci./Engrg. Computing Academy of Mathematics and System
More informationA Robust Preconditioned Iterative Method for the Navier-Stokes Equations with High Reynolds Numbers
Applied and Computational Mathematics 2017; 6(4): 202-207 http://www.sciencepublishinggroup.com/j/acm doi: 10.11648/j.acm.20170604.18 ISSN: 2328-5605 (Print); ISSN: 2328-5613 (Online) A Robust Preconditioned
More informationDepartment of Computer Science, University of Illinois at Urbana-Champaign
Department of Computer Science, University of Illinois at Urbana-Champaign Probing for Schur Complements and Preconditioning Generalized Saddle-Point Problems Eric de Sturler, sturler@cs.uiuc.edu, http://www-faculty.cs.uiuc.edu/~sturler
More informationOn the loss of orthogonality in the Gram-Schmidt orthogonalization process
CERFACS Technical Report No. TR/PA/03/25 Luc Giraud Julien Langou Miroslav Rozložník On the loss of orthogonality in the Gram-Schmidt orthogonalization process Abstract. In this paper we study numerical
More informationKey words. conjugate gradients, normwise backward error, incremental norm estimation.
Proceedings of ALGORITMY 2016 pp. 323 332 ON ERROR ESTIMATION IN THE CONJUGATE GRADIENT METHOD: NORMWISE BACKWARD ERROR PETR TICHÝ Abstract. Using an idea of Duff and Vömel [BIT, 42 (2002), pp. 300 322
More informationOUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU
Preconditioning Techniques for Solving Large Sparse Linear Systems Arnold Reusken Institut für Geometrie und Praktische Mathematik RWTH-Aachen OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative
More informationPreface to the Second Edition. Preface to the First Edition
n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................
More informationM.A. Botchev. September 5, 2014
Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev
More informationContribution of Wo¹niakowski, Strako²,... The conjugate gradient method in nite precision computa
Contribution of Wo¹niakowski, Strako²,... The conjugate gradient method in nite precision computations ªaw University of Technology Institute of Mathematics and Computer Science Warsaw, October 7, 2006
More informationSolving large sparse Ax = b.
Bob-05 p.1/69 Solving large sparse Ax = b. Stopping criteria, & backward stability of MGS-GMRES. Chris Paige (McGill University); Miroslav Rozložník & Zdeněk Strakoš (Academy of Sciences of the Czech Republic)..pdf
More informationCombination Preconditioning of saddle-point systems for positive definiteness
Combination Preconditioning of saddle-point systems for positive definiteness Andy Wathen Oxford University, UK joint work with Jen Pestana Eindhoven, 2012 p.1/30 compute iterates with residuals Krylov
More informationA Review of Preconditioning Techniques for Steady Incompressible Flow
Zeist 2009 p. 1/43 A Review of Preconditioning Techniques for Steady Incompressible Flow David Silvester School of Mathematics University of Manchester Zeist 2009 p. 2/43 PDEs Review : 1984 2005 Update
More informationOn the interplay between discretization and preconditioning of Krylov subspace methods
On the interplay between discretization and preconditioning of Krylov subspace methods Josef Málek and Zdeněk Strakoš Nečas Center for Mathematical Modeling Charles University in Prague and Czech Academy
More information9.1 Preconditioned Krylov Subspace Methods
Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete
More informationRegularized HSS iteration methods for saddle-point linear systems
BIT Numer Math DOI 10.1007/s10543-016-0636-7 Regularized HSS iteration methods for saddle-point linear systems Zhong-Zhi Bai 1 Michele Benzi 2 Received: 29 January 2016 / Accepted: 20 October 2016 Springer
More informationModified HSS iteration methods for a class of complex symmetric linear systems
Computing (2010) 87:9 111 DOI 10.1007/s00607-010-0077-0 Modified HSS iteration methods for a class of complex symmetric linear systems Zhong-Zhi Bai Michele Benzi Fang Chen Received: 20 October 2009 /
More informationIterative Methods and Multigrid
Iterative Methods and Multigrid Part 3: Preconditioning 2 Eric de Sturler Preconditioning The general idea behind preconditioning is that convergence of some method for the linear system Ax = b can be
More informationMathematics and Computer Science
Technical Report TR-2007-002 Block preconditioning for saddle point systems with indefinite (1,1) block by Michele Benzi, Jia Liu Mathematics and Computer Science EMORY UNIVERSITY International Journal
More informationParallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1
Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations
More informationPerformance Comparison of Relaxation Methods with Singular and Nonsingular Preconditioners for Singular Saddle Point Problems
Applied Mathematical Sciences, Vol. 10, 2016, no. 30, 1477-1488 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2016.6269 Performance Comparison of Relaxation Methods with Singular and Nonsingular
More informationLanczos tridigonalization and Golub - Kahan bidiagonalization: Ideas, connections and impact
Lanczos tridigonalization and Golub - Kahan bidiagonalization: Ideas, connections and impact Zdeněk Strakoš Academy of Sciences and Charles University, Prague http://www.cs.cas.cz/ strakos Hong Kong, February
More informationIndefinite Preconditioners for PDE-constrained optimization problems. V. Simoncini
Indefinite Preconditioners for PDE-constrained optimization problems V. Simoncini Dipartimento di Matematica, Università di Bologna, Italy valeria.simoncini@unibo.it Partly joint work with Debora Sesana,
More informationThe antitriangular factorisation of saddle point matrices
The antitriangular factorisation of saddle point matrices J. Pestana and A. J. Wathen August 29, 2013 Abstract Mastronardi and Van Dooren [this journal, 34 (2013) pp. 173 196] recently introduced the block
More informationON THE GENERALIZED DETERIORATED POSITIVE SEMI-DEFINITE AND SKEW-HERMITIAN SPLITTING PRECONDITIONER *
Journal of Computational Mathematics Vol.xx, No.x, 2x, 6. http://www.global-sci.org/jcm doi:?? ON THE GENERALIZED DETERIORATED POSITIVE SEMI-DEFINITE AND SKEW-HERMITIAN SPLITTING PRECONDITIONER * Davod
More informationIterative solution of saddle point problems
Iterative solution of saddle point problems Miroslav Rozložník, Pavel Jiránek Institute of Computer Science, Czech Academy of Sciences, Prague and Faculty of Mechatronics and Interdisciplinary Engineering
More informationOn the Superlinear Convergence of MINRES. Valeria Simoncini and Daniel B. Szyld. Report January 2012
On the Superlinear Convergence of MINRES Valeria Simoncini and Daniel B. Szyld Report 12-01-11 January 2012 This report is available in the World Wide Web at http://www.math.temple.edu/~szyld 0 Chapter
More informationPreconditioned inverse iteration and shift-invert Arnoldi method
Preconditioned inverse iteration and shift-invert Arnoldi method Melina Freitag Department of Mathematical Sciences University of Bath CSC Seminar Max-Planck-Institute for Dynamics of Complex Technical
More informationPreconditioners for reduced saddle point systems arising in elliptic PDE-constrained optimization problems
Zeng et al. Journal of Inequalities and Applications 205 205:355 DOI 0.86/s3660-05-0879-x RESEARCH Open Access Preconditioners for reduced saddle point systems arising in elliptic PDE-constrained optimization
More informationContents. Preface... xi. Introduction...
Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures... 1 1.1. Different types of parallelism... 1 1.1.1. Overlap, concurrency and parallelism... 1 1.1.2. Temporal and spatial parallelism
More informationarxiv: v1 [math.na] 26 Dec 2013
General constraint preconditioning iteration method for singular saddle-point problems Ai-Li Yang a,, Guo-Feng Zhang a, Yu-Jiang Wu a,b a School of Mathematics and Statistics, Lanzhou University, Lanzhou
More informationNUMERICS OF THE GRAM-SCHMIDT ORTHOGONALIZATION PROCESS
NUMERICS OF THE GRAM-SCHMIDT ORTHOGONALIZATION PROCESS Miro Rozložník Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic email: miro@cs.cas.cz joint results with Luc Giraud,
More informationIterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)
Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential
More informationON A GENERAL CLASS OF PRECONDITIONERS FOR NONSYMMETRIC GENERALIZED SADDLE POINT PROBLEMS
U..B. Sci. Bull., Series A, Vol. 78, Iss. 4, 06 ISSN 3-707 ON A GENERAL CLASS OF RECONDIIONERS FOR NONSYMMERIC GENERALIZED SADDLE OIN ROBLE Fatemeh anjeh Ali BEIK his paper deals with applying a class
More informationAn advanced ILU preconditioner for the incompressible Navier-Stokes equations
An advanced ILU preconditioner for the incompressible Navier-Stokes equations M. ur Rehman C. Vuik A. Segal Delft Institute of Applied Mathematics, TU delft The Netherlands Computational Methods with Applications,
More informationOn preconditioned MHSS iteration methods for complex symmetric linear systems
DOI 10.1007/s11075-010-9441-6 ORIGINAL PAPER On preconditioned MHSS iteration methods for complex symmetric linear systems Zhong-Zhi Bai Michele Benzi Fang Chen Received: 25 November 2010 / Accepted: 13
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationEfficient Augmented Lagrangian-type Preconditioning for the Oseen Problem using Grad-Div Stabilization
Efficient Augmented Lagrangian-type Preconditioning for the Oseen Problem using Grad-Div Stabilization Timo Heister, Texas A&M University 2013-02-28 SIAM CSE 2 Setting Stationary, incompressible flow problems
More informationNumerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
More informationSEMI-CONVERGENCE ANALYSIS OF THE INEXACT UZAWA METHOD FOR SINGULAR SADDLE POINT PROBLEMS
REVISTA DE LA UNIÓN MATEMÁTICA ARGENTINA Vol. 53, No. 1, 2012, 61 70 SEMI-CONVERGENCE ANALYSIS OF THE INEXACT UZAWA METHOD FOR SINGULAR SADDLE POINT PROBLEMS JIAN-LEI LI AND TING-ZHU HUANG Abstract. Recently,
More informationSome Preconditioning Techniques for Saddle Point Problems
Some Preconditioning Techniques for Saddle Point Problems Michele Benzi 1 and Andrew J. Wathen 2 1 Department of Mathematics and Computer Science, Emory University, Atlanta, Georgia 30322, USA. benzi@mathcs.emory.edu
More informationSpectral Properties of Saddle Point Linear Systems and Relations to Iterative Solvers Part I: Spectral Properties. V. Simoncini
Spectral Properties of Saddle Point Linear Systems and Relations to Iterative Solvers Part I: Spectral Properties V. Simoncini Dipartimento di Matematica, Università di ologna valeria@dm.unibo.it 1 Outline
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD
More informationA smooth variant of the fictitious domain approach
A smooth variant of the fictitious domain approach J. Haslinger, T. Kozubek, R. Kučera Department of umerical Mathematics, Charles University, Prague Department of Applied Mathematics, VŠB-TU, Ostrava
More informationLecture 18 Classical Iterative Methods
Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,
More information1 Error analysis for linear systems
Notes for 2016-09-16 1 Error analysis for linear systems We now discuss the sensitivity of linear systems to perturbations. This is relevant for two reasons: 1. Our standard recipe for getting an error
More informationOn solving linear systems arising from Shishkin mesh discretizations
On solving linear systems arising from Shishkin mesh discretizations Petr Tichý Faculty of Mathematics and Physics, Charles University joint work with Carlos Echeverría, Jörg Liesen, and Daniel Szyld October
More informationBlock triangular preconditioner for static Maxwell equations*
Volume 3, N. 3, pp. 589 61, 11 Copyright 11 SBMAC ISSN 11-85 www.scielo.br/cam Block triangular preconditioner for static Maxwell equations* SHI-LIANG WU 1, TING-ZHU HUANG and LIANG LI 1 School of Mathematics
More informationLinear and Non-Linear Preconditioning
and Non- martin.gander@unige.ch University of Geneva July 2015 Joint work with Victorita Dolean, Walid Kheriji, Felix Kwok and Roland Masson (1845): First Idea of After preconditioning, it takes only three
More informationON ORTHOGONAL REDUCTION TO HESSENBERG FORM WITH SMALL BANDWIDTH
ON ORTHOGONAL REDUCTION TO HESSENBERG FORM WITH SMALL BANDWIDTH V. FABER, J. LIESEN, AND P. TICHÝ Abstract. Numerous algorithms in numerical linear algebra are based on the reduction of a given matrix
More informationOn the Preconditioning of the Block Tridiagonal Linear System of Equations
On the Preconditioning of the Block Tridiagonal Linear System of Equations Davod Khojasteh Salkuyeh Department of Mathematics, University of Mohaghegh Ardabili, PO Box 179, Ardabil, Iran E-mail: khojaste@umaacir
More informationETNA Kent State University
Electronic Transactions on Numerical Analysis. Volume 36, pp. 39-53, 009-010. Copyright 009,. ISSN 1068-9613. ETNA P-REGULAR SPLITTING ITERATIVE METHODS FOR NON-HERMITIAN POSITIVE DEFINITE LINEAR SYSTEMS
More informationON AUGMENTED LAGRANGIAN METHODS FOR SADDLE-POINT LINEAR SYSTEMS WITH SINGULAR OR SEMIDEFINITE (1,1) BLOCKS * 1. Introduction
Journal of Computational Mathematics Vol.xx, No.x, 200x, 1 9. http://www.global-sci.org/jcm doi:10.4208/jcm.1401-cr7 ON AUGMENED LAGRANGIAN MEHODS FOR SADDLE-POIN LINEAR SYSEMS WIH SINGULAR OR SEMIDEFINIE
More informationON A SPLITTING PRECONDITIONER FOR SADDLE POINT PROBLEMS
J. Appl. Math. & Informatics Vol. 36(208, No. 5-6, pp. 459-474 https://doi.org/0.437/jami.208.459 ON A SPLITTING PRECONDITIONER FOR SADDLE POINT PROBLEMS DAVOD KHOJASTEH SALKUYEH, MARYAM ABDOLMALEKI, SAEED
More informationFast solvers for steady incompressible flow
ICFD 25 p.1/21 Fast solvers for steady incompressible flow Andy Wathen Oxford University wathen@comlab.ox.ac.uk http://web.comlab.ox.ac.uk/~wathen/ Joint work with: Howard Elman (University of Maryland,
More informationarxiv: v2 [math.na] 1 Feb 2013
A FRAMEWORK FOR DEFLATED AND AUGMENTED KRYLOV SUBSPACE METHODS ANDRÉ GAUL, MARTIN H. GUTKNECHT, JÖRG LIESEN AND REINHARD NABBEN arxiv:1206.1506v2 [math.na] 1 Feb 2013 Abstract. We consider deflation and
More informationEfficient Estimation of the A-norm of the Error in the Preconditioned Conjugate Gradient Method
Efficient Estimation of the A-norm of the Error in the Preconditioned Conjugate Gradient Method Zdeněk Strakoš and Petr Tichý Institute of Computer Science AS CR, Technical University of Berlin. Emory
More informationTwo efficient inexact algorithms for a class of large sparse complex linear systems
Two efficient inexact algorithms for a class of large sparse complex linear systems Vahid Edalatpour, Davod Hezari and Davod Khojasteh Salkuyeh Faculty of Mathematical Sciences, University of Guilan, Rasht,
More informationThe semi-convergence of GSI method for singular saddle point problems
Bull. Math. Soc. Sci. Math. Roumanie Tome 57(05 No., 04, 93 00 The semi-convergence of GSI method for singular saddle point problems by Shu-Xin Miao Abstract Recently, Miao Wang considered the GSI method
More informationNested splitting CG-like iterative method for solving the continuous Sylvester equation and preconditioning
Adv Comput Math DOI 10.1007/s10444-013-9330-3 Nested splitting CG-like iterative method for solving the continuous Sylvester equation and preconditioning Mohammad Khorsand Zak Faezeh Toutounian Received:
More informationEfficient Estimation of the A-norm of the Error in the Preconditioned Conjugate Gradient Method
Efficient Estimation of the A-norm of the Error in the Preconditioned Conjugate Gradient Method Zdeněk Strakoš and Petr Tichý Institute of Computer Science AS CR, Technical University of Berlin. International
More informationDELFT UNIVERSITY OF TECHNOLOGY
DELFT UNIVERSITY OF TECHNOLOGY REPORT 13-10 Comparison of some preconditioners for the incompressible Navier-Stokes equations X. He and C. Vuik ISSN 1389-6520 Reports of the Delft Institute of Applied
More informationIN this paper, we investigate spectral properties of block
On the Eigenvalues Distribution of Preconditioned Block wo-by-two Matrix Mu-Zheng Zhu and a-e Qi Abstract he spectral properties of a class of block matrix are studied, which arise in the numercial solutions
More informationIndefinite and physics-based preconditioning
Indefinite and physics-based preconditioning Jed Brown VAW, ETH Zürich 2009-01-29 Newton iteration Standard form of a nonlinear system F (u) 0 Iteration Solve: Update: J(ũ)u F (ũ) ũ + ũ + u Example (p-bratu)
More informationKey words. GMRES method, convergence bounds, worst-case GMRES, ideal GMRES, field of values
THE FIELD OF VALUES BOUNDS ON IDEAL GMRES JÖRG LIESEN AND PETR TICHÝ 27.03.2018) Abstract. A widely known result of Elman, and its improvements due to Starke, Eiermann and Ernst, gives a bound on the worst-case
More informationITERATIVE METHODS FOR SPARSE LINEAR SYSTEMS
ITERATIVE METHODS FOR SPARSE LINEAR SYSTEMS YOUSEF SAAD University of Minnesota PWS PUBLISHING COMPANY I(T)P An International Thomson Publishing Company BOSTON ALBANY BONN CINCINNATI DETROIT LONDON MADRID
More informationOn the influence of eigenvalues on Bi-CG residual norms
On the influence of eigenvalues on Bi-CG residual norms Jurjen Duintjer Tebbens Institute of Computer Science Academy of Sciences of the Czech Republic duintjertebbens@cs.cas.cz Gérard Meurant 30, rue
More information7.4 The Saddle Point Stokes Problem
346 CHAPTER 7. APPLIED FOURIER ANALYSIS 7.4 The Saddle Point Stokes Problem So far the matrix C has been diagonal no trouble to invert. This section jumps to a fluid flow problem that is still linear (simpler
More informationError Bounds for Iterative Refinement in Three Precisions
Error Bounds for Iterative Refinement in Three Precisions Erin C. Carson, New York University Nicholas J. Higham, University of Manchester SIAM Annual Meeting Portland, Oregon July 13, 018 Hardware Support
More informationRecent advances in approximation using Krylov subspaces. V. Simoncini. Dipartimento di Matematica, Università di Bologna.
Recent advances in approximation using Krylov subspaces V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1 The framework It is given an operator
More informationJournal of Computational and Applied Mathematics. Multigrid method for solving convection-diffusion problems with dominant convection
Journal of Computational and Applied Mathematics 226 (2009) 77 83 Contents lists available at ScienceDirect Journal of Computational and Applied Mathematics journal homepage: www.elsevier.com/locate/cam
More informationLecture 3: Inexact inverse iteration with preconditioning
Lecture 3: Department of Mathematical Sciences CLAPDE, Durham, July 2008 Joint work with M. Freitag (Bath), and M. Robbé & M. Sadkane (Brest) 1 Introduction 2 Preconditioned GMRES for Inverse Power Method
More informationA MODIFIED HSS ITERATION METHOD FOR SOLVING THE COMPLEX LINEAR MATRIX EQUATION AXB = C *
Journal of Computational Mathematics Vol.34, No.4, 2016, 437 450. http://www.global-sci.org/jcm doi:10.4208/jcm.1601-m2015-0416 A MODIFIED HSS ITERATION METHOD FOR SOLVING THE COMPLEX LINEAR MATRIX EQUATION
More informationMS 28: Scalable Communication-Avoiding and -Hiding Krylov Subspace Methods I
MS 28: Scalable Communication-Avoiding and -Hiding Krylov Subspace Methods I Organizers: Siegfried Cools, University of Antwerp, Belgium Erin C. Carson, New York University, USA 10:50-11:10 High Performance
More informationAdaptive version of Simpler GMRES
DOI 0.007/s075-009-93- ORIGINAL PAPER Adaptive version of Simpler GMRES Pavel Jiránek Miroslav Rozložník Received: 0 November 008 / Accepted: 3 June 009 Springer Science + Business Media, LLC 009 Abstract
More informationBlock-Triangular and Skew-Hermitian Splitting Methods for Positive Definite Linear Systems
Block-Triangular and Skew-Hermitian Splitting Methods for Positive Definite Linear Systems Zhong-Zhi Bai State Key Laboratory of Scientific/Engineering Computing Institute of Computational Mathematics
More informationConvergence Properties of Preconditioned Hermitian and Skew-Hermitian Splitting Methods for Non-Hermitian Positive Semidefinite Matrices
Convergence Properties of Preconditioned Hermitian and Skew-Hermitian Splitting Methods for Non-Hermitian Positive Semidefinite Matrices Zhong-Zhi Bai 1 Department of Mathematics, Fudan University Shanghai
More informationIterative methods for Linear System
Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and
More informationKey words. inf-sup constant, iterative solvers, preconditioning, saddle point problems
NATURAL PRECONDITIONING AND ITERATIVE METHODS FOR SADDLE POINT SYSTEMS JENNIFER PESTANA AND ANDREW J. WATHEN Abstract. The solution of quadratic or locally quadratic extremum problems subject to linear(ized)
More informationIn order to solve the linear system KL M N when K is nonsymmetric, we can solve the equivalent system
!"#$% "&!#' (%)!#" *# %)%(! #! %)!#" +, %"!"#$ %*&%! $#&*! *# %)%! -. -/ 0 -. 12 "**3! * $!#%+,!2!#% 44" #% ! # 4"!#" "%! "5"#!!#6 -. - #% " 7% "3#!#3! - + 87&2! * $!#% 44" ) 3( $! # % %#!!#%+ 9332!
More informationCONVERGENCE BOUNDS FOR PRECONDITIONED GMRES USING ELEMENT-BY-ELEMENT ESTIMATES OF THE FIELD OF VALUES
European Conference on Computational Fluid Dynamics ECCOMAS CFD 2006 P. Wesseling, E. Oñate and J. Périaux (Eds) c TU Delft, The Netherlands, 2006 CONVERGENCE BOUNDS FOR PRECONDITIONED GMRES USING ELEMENT-BY-ELEMENT
More informationInexact inverse iteration with preconditioning
Department of Mathematical Sciences Computational Methods with Applications Harrachov, Czech Republic 24th August 2007 (joint work with M. Robbé and M. Sadkane (Brest)) 1 Introduction 2 Preconditioned
More informationIterative solvers for saddle point algebraic linear systems: tools of the trade. V. Simoncini
Iterative solvers for saddle point algebraic linear systems: tools of the trade V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1 The problem
More informationIterative Solution methods
p. 1/28 TDB NLA Parallel Algorithms for Scientific Computing Iterative Solution methods p. 2/28 TDB NLA Parallel Algorithms for Scientific Computing Basic Iterative Solution methods The ideas to use iterative
More information