# Recent advances in approximation using Krylov subspaces. V. Simoncini. Dipartimento di Matematica, Università di Bologna.

Size: px
Start display at page:

## Transcription

1 Recent advances in approximation using Krylov subspaces V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy 1

2 The framework It is given an operator v A ɛ (v). Efficiently solve the given problem in the approximation space K m = span{v, A ɛ1 (v), A ɛ2 (A ɛ1 (v)),...}, v C n with dim(k m ) = m, where A ɛ A for ɛ 0 (ɛ may be tuned) for A = A, ɛ = 0 K m = span{v, Av, A 2 v,..., A m 1 v} 2

3 Examples of A: Solution of (preconditioned) large linear systems, Ax = b n n A = A Shift-and-Invert eigensolvers Ax = λmx, x = 1, A = (σm A) 1 Spectral Transformation for the exponential x = exp(a)v, A = (γi A) 1... Goal: Achieve approximation x m to x within a fixed tolerance, by using A ɛ (and not A), with variable ɛ 3

4 Many applications in Scientific Computing A(v) function (linear in v): Shift-and-Invert procedures for interior eigenvalues Schur complement: A = B T S 1 B S expensive to invert Preconditioned system: AP 1 x = b, where etc. P 1 v i P 1 i v i K m = span{v, A(v), A(A(v)),...}, v C n 4

5 The exact approach To focus our attention: A = A. Krylov subspace: K m = span{v, Av, A 2 v,..., A m 1 v} V m = [v 1,..., v m ] orthogonal basis 1. v m+1 = Av m 2. v m+1 orthogonalize v m+1 w.r.to {v i } Key relation in Krylov subspace methods: AV m = V m H m + v m+1 h m+1,m e T m v = V m e 1 v 5

6 The exact approach. cont d K m Krylov subspace V m = [v 1,..., v m ] orthogonal basis AV m = V m H m + v m+1 h m+1,m e T m = V m H m with v = V m e 1 v System: x m K m x m = V m y m (x 0 = 0) Residual r m = v Ax m = V m+1 (e 1 v H m y m ) Eigenpb: (θ, y) eigenpair of H m (θ, V m y) Ritz pair for (λ, x) Residual: r m = θv m y AV m y = v m+1 h m+1,m e T my 6

7 AV m = V m+1 H m + The inexact key relation A = A A ɛ (v) = Av + f F m }{{} [f 1,f 2,...,f m ] How large is F m allowed to be? system: F m error matrix, f j = O(ɛ j ) r m = b AV m y m = b V m+1 H m y m F m y m eigenproblem: (θ, V m y) = V m+1 (e 1 β H m y m ) F m y m }{{} computed residual =: r m r m = θv m y AV m y = v m+1 h m+1,m e T my F m y 7

8 A dynamic setting F m y = [f 1, f 2,..., f m ] η 1 η 2. = m f i η i i=1 η m The terms f i η i need to be small: f i η i < 1 m ɛ i F my < ɛ If η i small f i is allowed to be large 8

9 Linear systems: The solution pattern y m = [η 1 ; η 2 ;... ; η m ] depends on the chosen method, e.g. Petrov-Galerkin (e.g. GMRES): y m = argmin y e 1 β H m y, η i 1 σ min (H m ) r i 1 r i 1 : GMRES computed residual at iteration i 1. Simoncini & Szyld, SISC 2003 (see also Sleijpen & van den Eshof, SIMAX 2004) Analogous result for Galerkin methods (e.g. FOM) 9

10 Ritz approximation: Eigenproblem: The structure of the Ritz pair (θ, y) eigenpair of H m y = [η 1 ; η 2 ;... ; η m ], η i 2 δ m,i r i 1 δ m,i quantity related to the spectral gap of θ with H m r i 1 : Computed eigenresidual at iteration i 1 Analogous results for Harmonic Ritz values and Lanczos approx. Simoncini, SINUM

11 Relaxing the inexactness in A A v i not performed exactly (A + E i ) v i True (unobservable) vs. computed residuals: r m = b AV m y m = V m+1 (e 1 β H m y m ) F m y m - GMRES: If (Similar result for FOM) E i σ min(h m ) m 1 r i 1 ε i = 1,..., m then F m y m ε r m V m+1 (e 1 β H m y m ) ε r i 1 : GMRES computed residual at iteration i 1 11

12 An example: Schur complement } B T S {{ 1 B } x = b A y i B T S 1 Bv i At each Krylov subspace iteration: 8 < Solve Sw i = Bv i : Compute y i = B T w i Inexact 8 < Approx solve Sw i = Bv i : Compute by i = B T bw i bw i w i = ŵ i + ɛ i ɛ i error in inner solution so that Av i B T ŵ i = B T w }{{} i B T ɛ i = (A + E }{{} i )v i Av i E i v i 12

13 Numerical experiment } B T S {{ 1 B } x = b at each it. i solve Sw i = Bv i A r m 10 2 ε inner Inexact FOM δ m = r m (b V m+1 H m y m ) magnitude δ m 10 7 ~ r m number of iterations 13

14 Eigenproblem Inverted Arnoldi: Ax = λx Find min λ y A(v) = A 1 v Matrix sherman inner residual norm imaginary part 0 magnitude ε exact res. inexact residual real part number of outer iterations 14

15 Problems to be faced Make the inexactness criterion practical E i σ min(h m ) m 1 r i 1 ε E i l m 1 r i 1 ε (CERFACS tr s of Bouras, Frayssè, Giraud, 2000, Bouras, Frayssè SIMAX05) What is the convergence behavior? What if original A was symmetric? 15

16 Selecting l m : system AP 1 x = b ε inner ε inner magnitude 10 6 r m magnitude 10 6 r m δ m ~ r m δ m ~ r m number of iterations number of iterations Left: l m = 1 Right: estimated l m Final Requested Tolerance:

17 Convergence behavior Does the inexact procedure behave as if E i = 0? The Sleijpen & van den Eshof s example: Exact vs. Inexact GMRES b = e 1 E i random entries E k A = residual norm inexact number of iterations exact 17

18 Inexactness and convergence (linear systems) Av i (A + E i )v i For general A and b convergence is the same as exact A Problems for: Sensitive A (highly nonnormal) Special starting vector / right-hand side Superlinear convergence as for A (Simoncini & Szyld, SIREV 2005) 18

19 Flexible preconditioning AP 1 x = b x = P 1 x Flexible: P 1 v i P 1 i v i, bx m span{v 1, AP 1 1 v 1, AP 1 2 v 2,..., AP 1 m 1 v m 1} Directly recover x m (Saad, 1993): [P 1 1 v 1, P 1 2 v 2,..., P 1 m v m ] = Z m x m = Z m y m Inexact framework but exact residual 19

20 A practical example A B B T 0 x y = f g P = I 0 0 B T B Application of P 1 corresponds to solves with B T B P Use CG to solve systems with B T B Variable inner tolerance: At each outer iteration m, r inner k l m r outer m 1 ε 20

21 Outer tolerance: 10 8 Electromagnetic 2D problem r inner k l m r outer m 1 ε 0 ε Elapsed Time Pb. Size Fixed Inner Tol Var. Inner Tol. Var. Inner Tol. ε = ε = / r ε = / r (54) 11.4 (54) 14.7 (54) (58) 62.8 (58) 70.7 (58) (54) (54) (54) 21

22 Structural Dynamics (A + σb)x = b Solve for many σ s simultaneously (AB 1 + σi) x = b (Perotti & Simoncini 2002) Inexact solutions with B at each iteration: Prec. Fill-in 5 Prec. Fill-in 10 e-time [s] # outer its e-time [s] # outer its Tol Dynamic Tol % enhancement with tiny change in the code (see also van den Eshof, Sleijpen and van Gijzen, JCAM 2005) 22

23 Inexactness when A symmetric A symmetric A + E i nonsymmetric Assume V T m V m = I H m upper Hessenberg Wise implementation of short-term recurr. /truncated methods (V m non-orth. W m, H m tridiag./banded T m ) - Inexact short-term recurrence system solvers (Golub-Overton 88, Golub-Ye 99, Notay 00, Sleijpen-van den Eshof 04,... ) - Inexact symmetric eigensolvers (Lai-Lin-Lin 1997, Golub-Ye 2000, Golub-Zhang-Zha 2000, Notay 2002,... ) - Truncated methods (Simoncini - Szyld, Numer.Math. 2005) 23

24 Ax = b A sym. (2D Laplacian) Preconditioner: P nonsymmetric perturbation (10 5 ) of Incomplete Cholesky ^ T m+1,m T m+1,m 10 4 PCG 10 4 norm of residual 10 6 magnitude * I W W m+1 m+1 m k = k = 2, number of iterations number of iterations 24

25 One more application: Approximation of the exponential A symmetric negative semidefinite (large dimension), v s.t. v = 1, exp(a)v x m = V m y m, y m = exp(h m )e 1 Problem: Find acceleration process for A to speed up convergence Hochbruck & van den Eshof (SISC 2006): Determine x m exp(a)v as x m = V m y m K m ((γi A) 1, v) for some scalar γ y m = exp(h m )e 1 has a structured decreasing pattern (Lopez & Simoncini, SINUM 2006) 25

26 Conclusions A may be replaced by A ɛi with increasing ɛ i and still converge Stable procedure for not too sensitive (e.g. non-normal) problems Property inherent of Krylov approximation Many more applications for this general setting References at: simoncin 26

### The Nullspace free eigenvalue problem and the inexact Shift and invert Lanczos method. V. Simoncini. Dipartimento di Matematica, Università di Bologna

The Nullspace free eigenvalue problem and the inexact Shift and invert Lanczos method V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1 The

### Inexactness and flexibility in linear Krylov solvers

Inexactness and flexibility in linear Krylov solvers Luc Giraud ENSEEIHT (N7) - IRIT, Toulouse Matrix Analysis and Applications CIRM Luminy - October 15-19, 2007 in honor of Gérard Meurant for his 60 th

### Approximation of functions of large matrices. Part I. Computational aspects. V. Simoncini

Approximation of functions of large matrices. Part I. Computational aspects V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1 The problem Given

### Inexact inverse iteration with preconditioning

Department of Mathematical Sciences Computational Methods with Applications Harrachov, Czech Republic 24th August 2007 (joint work with M. Robbé and M. Sadkane (Brest)) 1 Introduction 2 Preconditioned

### Preconditioned inverse iteration and shift-invert Arnoldi method

Preconditioned inverse iteration and shift-invert Arnoldi method Melina Freitag Department of Mathematical Sciences University of Bath CSC Seminar Max-Planck-Institute for Dynamics of Complex Technical

### Lecture 3: Inexact inverse iteration with preconditioning

Lecture 3: Department of Mathematical Sciences CLAPDE, Durham, July 2008 Joint work with M. Freitag (Bath), and M. Robbé & M. Sadkane (Brest) 1 Introduction 2 Preconditioned GMRES for Inverse Power Method

### Iterative solvers for saddle point algebraic linear systems: tools of the trade. V. Simoncini

Iterative solvers for saddle point algebraic linear systems: tools of the trade V. Simoncini Dipartimento di Matematica, Università di Bologna and CIRSA, Ravenna, Italy valeria@dm.unibo.it 1 The problem

### 1. Introduction. In this paper we consider the large and sparse eigenvalue problem. Ax = λx (1.1) T (λ)x = 0 (1.2)

A NEW JUSTIFICATION OF THE JACOBI DAVIDSON METHOD FOR LARGE EIGENPROBLEMS HEINRICH VOSS Abstract. The Jacobi Davidson method is known to converge at least quadratically if the correction equation is solved

### M.A. Botchev. September 5, 2014

Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev

### Adaptive rational Krylov subspaces for large-scale dynamical systems. V. Simoncini

Adaptive rational Krylov subspaces for large-scale dynamical systems V. Simoncini Dipartimento di Matematica, Università di Bologna valeria@dm.unibo.it joint work with Vladimir Druskin, Schlumberger Doll

### Relaxation strategies for nested Krylov methods

Journal of Computational and Applied Mathematics 177 (2005) 347 365 www.elsevier.com/locate/cam Relaxation strategies for nested Krylov methods Jasper van den Eshof a, Gerard L.G. Sleijpen b,, Martin B.

### On the Superlinear Convergence of MINRES. Valeria Simoncini and Daniel B. Szyld. Report January 2012

On the Superlinear Convergence of MINRES Valeria Simoncini and Daniel B. Szyld Report 12-01-11 January 2012 This report is available in the World Wide Web at http://www.math.temple.edu/~szyld 0 Chapter

### Iterative Methods for Linear Systems of Equations

Iterative Methods for Linear Systems of Equations Projection methods (3) ITMAN PhD-course DTU 20-10-08 till 24-10-08 Martin van Gijzen 1 Delft University of Technology Overview day 4 Bi-Lanczos method

### LARGE SPARSE EIGENVALUE PROBLEMS. General Tools for Solving Large Eigen-Problems

LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization General Tools for Solving Large Eigen-Problems

### Eigenvalue Problems CHAPTER 1 : PRELIMINARIES

Eigenvalue Problems CHAPTER 1 : PRELIMINARIES Heinrich Voss voss@tu-harburg.de Hamburg University of Technology Institute of Mathematics TUHH Heinrich Voss Preliminaries Eigenvalue problems 2012 1 / 14

### LARGE SPARSE EIGENVALUE PROBLEMS

LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization 14-1 General Tools for Solving Large Eigen-Problems

### Finding Rightmost Eigenvalues of Large, Sparse, Nonsymmetric Parameterized Eigenvalue Problems

Finding Rightmost Eigenvalues of Large, Sparse, Nonsymmetric Parameterized Eigenvalue Problems AMSC 663-664 Final Report Minghao Wu AMSC Program mwu@math.umd.edu Dr. Howard Elman Department of Computer

### Iterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)

Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential

### ABSTRACT NUMERICAL SOLUTION OF EIGENVALUE PROBLEMS WITH SPECTRAL TRANSFORMATIONS

ABSTRACT Title of dissertation: NUMERICAL SOLUTION OF EIGENVALUE PROBLEMS WITH SPECTRAL TRANSFORMATIONS Fei Xue, Doctor of Philosophy, 2009 Dissertation directed by: Professor Howard C. Elman Department

### RAYLEIGH QUOTIENT ITERATION AND SIMPLIFIED JACOBI-DAVIDSON WITH PRECONDITIONED ITERATIVE SOLVES FOR GENERALISED EIGENVALUE PROBLEMS

RAYLEIGH QUOTIENT ITERATION AND SIMPLIFIED JACOBI-DAVIDSON WITH PRECONDITIONED ITERATIVE SOLVES FOR GENERALISED EIGENVALUE PROBLEMS MELINA A. FREITAG, ALASTAIR SPENCE, AND EERO VAINIKKO Abstract. The computation

### Is there life after the Lanczos method? What is LOBPCG?

1 Is there life after the Lanczos method? What is LOBPCG? Andrew V. Knyazev Department of Mathematics and Center for Computational Mathematics University of Colorado at Denver SIAM ALA Meeting, July 17,

### ABSTRACT OF DISSERTATION. Ping Zhang

ABSTRACT OF DISSERTATION Ping Zhang The Graduate School University of Kentucky 2009 Iterative Methods for Computing Eigenvalues and Exponentials of Large Matrices ABSTRACT OF DISSERTATION A dissertation

### Iterative methods for Linear System

Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and

### EIGIFP: A MATLAB Program for Solving Large Symmetric Generalized Eigenvalue Problems

EIGIFP: A MATLAB Program for Solving Large Symmetric Generalized Eigenvalue Problems JAMES H. MONEY and QIANG YE UNIVERSITY OF KENTUCKY eigifp is a MATLAB program for computing a few extreme eigenvalues

### Alternative correction equations in the Jacobi-Davidson method

Chapter 2 Alternative correction equations in the Jacobi-Davidson method Menno Genseberger and Gerard Sleijpen Abstract The correction equation in the Jacobi-Davidson method is effective in a subspace

### The rational Krylov subspace for parameter dependent systems. V. Simoncini

The rational Krylov subspace for parameter dependent systems V. Simoncini Dipartimento di Matematica, Università di Bologna valeria.simoncini@unibo.it 1 Given the continuous-time system Motivation. Model

### IDR(s) as a projection method

Delft University of Technology Faculty of Electrical Engineering, Mathematics and Computer Science Delft Institute of Applied Mathematics IDR(s) as a projection method A thesis submitted to the Delft Institute

### Valeria Simoncini and Daniel B. Szyld. Report October 2009

Interpreting IDR as a Petrov-Galerkin method Valeria Simoncini and Daniel B. Szyld Report 09-10-22 October 2009 This report is available in the World Wide Web at http://www.math.temple.edu/~szyld INTERPRETING

### c 2006 Society for Industrial and Applied Mathematics

SIAM J. MATRIX ANAL. APPL. Vol. 28, No. 4, pp. 1069 1082 c 2006 Society for Industrial and Applied Mathematics INEXACT INVERSE ITERATION WITH VARIABLE SHIFT FOR NONSYMMETRIC GENERALIZED EIGENVALUE PROBLEMS

### Inner-outer Iterative Methods for Eigenvalue Problems - Convergence and Preconditioning

Inner-outer Iterative Methods for Eigenvalue Problems - Convergence and Preconditioning submitted by Melina Annerose Freitag for the degree of Doctor of Philosophy of the University of Bath Department

### Solution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method

Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts

### A Model-Trust-Region Framework for Symmetric Generalized Eigenvalue Problems

A Model-Trust-Region Framework for Symmetric Generalized Eigenvalue Problems C. G. Baker P.-A. Absil K. A. Gallivan Technical Report FSU-SCS-2005-096 Submitted June 7, 2005 Abstract A general inner-outer

### CONTROLLING INNER ITERATIONS IN THE JACOBI DAVIDSON METHOD

CONTROLLING INNER ITERATIONS IN THE JACOBI DAVIDSON METHOD MICHIEL E. HOCHSTENBACH AND YVAN NOTAY Abstract. The Jacobi Davidson method is an eigenvalue solver which uses the iterative (and in general inaccurate)

### Recent computational developments in Krylov Subspace Methods for linear systems. Valeria Simoncini and Daniel B. Szyld

Recent computational developments in Krylov Subspace Methods for linear systems Valeria Simoncini and Daniel B. Szyld A later version appeared in : Numerical Linear Algebra w/appl., 2007, v. 14(1), pp.1-59.

### c 2004 Society for Industrial and Applied Mathematics

SIAM J. MATRIX ANAL. APPL. Vol. 6, No., pp. 377 389 c 004 Society for Industrial and Applied Mathematics SPECTRAL PROPERTIES OF THE HERMITIAN AND SKEW-HERMITIAN SPLITTING PRECONDITIONER FOR SADDLE POINT

### Controlling inner iterations in the Jacobi Davidson method

Controlling inner iterations in the Jacobi Davidson method Michiel E. Hochstenbach and Yvan Notay Service de Métrologie Nucléaire, Université Libre de Bruxelles (C.P. 165/84), 50, Av. F.D. Roosevelt, B-1050

### CONTROLLING INNER ITERATIONS IN THE JACOBI DAVIDSON METHOD

CONTROLLING INNER ITERATIONS IN THE JACOBI DAVIDSON METHOD MICHIEL E. HOCHSTENBACH AND YVAN NOTAY Abstract. The Jacobi Davidson method is an eigenvalue solver which uses an inner-outer scheme. In the outer

### Nested Krylov methods for shifted linear systems

Nested Krylov methods for shifted linear systems M. Baumann, and M. B. van Gizen Email: M.M.Baumann@tudelft.nl Delft Institute of Applied Mathematics Delft University of Technology Delft, The Netherlands

### A Tuned Preconditioner for Inexact Inverse Iteration Applied to Hermitian Eigenvalue Problems

A Tuned Preconditioner for Applied to Eigenvalue Problems Department of Mathematical Sciences University of Bath, United Kingdom IWASEP VI May 22-25, 2006 Pennsylvania State University, University Park

### A Tuned Preconditioner for Inexact Inverse Iteration for Generalised Eigenvalue Problems

A Tuned Preconditioner for for Generalised Eigenvalue Problems Department of Mathematical Sciences University of Bath, United Kingdom IWASEP VI May 22-25, 2006 Pennsylvania State University, University

### State-of-the-art numerical solution of large Hermitian eigenvalue problems. Andreas Stathopoulos

State-of-the-art numerical solution of large Hermitian eigenvalue problems Andreas Stathopoulos Computer Science Department and Computational Sciences Cluster College of William and Mary Acknowledgment:

### Davidson Method CHAPTER 3 : JACOBI DAVIDSON METHOD

Davidson Method CHAPTER 3 : JACOBI DAVIDSON METHOD Heinrich Voss voss@tu-harburg.de Hamburg University of Technology The Davidson method is a popular technique to compute a few of the smallest (or largest)

### Order reduction numerical methods for the algebraic Riccati equation. V. Simoncini

Order reduction numerical methods for the algebraic Riccati equation V. Simoncini Dipartimento di Matematica Alma Mater Studiorum - Università di Bologna valeria.simoncini@unibo.it 1 The problem Find X

### Jacobi s Ideas on Eigenvalue Computation in a modern context

Jacobi s Ideas on Eigenvalue Computation in a modern context Henk van der Vorst vorst@math.uu.nl Mathematical Institute Utrecht University June 3, 2006, Michel Crouzeix p.1/18 General remarks Ax = λx Nonlinear

### Applied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic

Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.4: Krylov Subspace Methods 2 / 51 Krylov Subspace Methods In this chapter we give

### Charles University Faculty of Mathematics and Physics DOCTORAL THESIS. Krylov subspace approximations in linear algebraic problems

Charles University Faculty of Mathematics and Physics DOCTORAL THESIS Iveta Hnětynková Krylov subspace approximations in linear algebraic problems Department of Numerical Mathematics Supervisor: Doc. RNDr.

### A Robust Preconditioned Iterative Method for the Navier-Stokes Equations with High Reynolds Numbers

Applied and Computational Mathematics 2017; 6(4): 202-207 http://www.sciencepublishinggroup.com/j/acm doi: 10.11648/j.acm.20170604.18 ISSN: 2328-5605 (Print); ISSN: 2328-5613 (Online) A Robust Preconditioned

### Preconditioned Locally Minimal Residual Method for Computing Interior Eigenpairs of Symmetric Operators

Preconditioned Locally Minimal Residual Method for Computing Interior Eigenpairs of Symmetric Operators Eugene Vecharynski 1 Andrew Knyazev 2 1 Department of Computer Science and Engineering University

### DELFT UNIVERSITY OF TECHNOLOGY

DELFT UNIVERSITY OF TECHNOLOGY REPORT 16-02 The Induced Dimension Reduction method applied to convection-diffusion-reaction problems R. Astudillo and M. B. van Gijzen ISSN 1389-6520 Reports of the Delft

### A Jacobi Davidson Method for Nonlinear Eigenproblems

A Jacobi Davidson Method for Nonlinear Eigenproblems Heinrich Voss Section of Mathematics, Hamburg University of Technology, D 21071 Hamburg voss @ tu-harburg.de http://www.tu-harburg.de/mat/hp/voss Abstract.

### Numerical behavior of inexact linear solvers

Numerical behavior of inexact linear solvers Miro Rozložník joint results with Zhong-zhi Bai and Pavel Jiránek Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic The fourth

### Arnoldi Methods in SLEPc

Scalable Library for Eigenvalue Problem Computations SLEPc Technical Report STR-4 Available at http://slepc.upv.es Arnoldi Methods in SLEPc V. Hernández J. E. Román A. Tomás V. Vidal Last update: October,

### A HARMONIC RESTARTED ARNOLDI ALGORITHM FOR CALCULATING EIGENVALUES AND DETERMINING MULTIPLICITY

A HARMONIC RESTARTED ARNOLDI ALGORITHM FOR CALCULATING EIGENVALUES AND DETERMINING MULTIPLICITY RONALD B. MORGAN AND MIN ZENG Abstract. A restarted Arnoldi algorithm is given that computes eigenvalues

### On the accuracy of saddle point solvers

On the accuracy of saddle point solvers Miro Rozložník joint results with Valeria Simoncini and Pavel Jiránek Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic Seminar at

### Last Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection

Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue

### Preface to the Second Edition. Preface to the First Edition

n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................

### Fast iterative solvers

Utrecht, 15 november 2017 Fast iterative solvers Gerard Sleijpen Department of Mathematics http://www.staff.science.uu.nl/ sleij101/ Review. To simplify, take x 0 = 0, assume b 2 = 1. Solving Ax = b for

### On prescribing Ritz values and GMRES residual norms generated by Arnoldi processes

On prescribing Ritz values and GMRES residual norms generated by Arnoldi processes Jurjen Duintjer Tebbens Institute of Computer Science Academy of Sciences of the Czech Republic joint work with Gérard

### ITERATIVE PROJECTION METHODS FOR SPARSE LINEAR SYSTEMS AND EIGENPROBLEMS CHAPTER 11 : JACOBI DAVIDSON METHOD

ITERATIVE PROJECTION METHODS FOR SPARSE LINEAR SYSTEMS AND EIGENPROBLEMS CHAPTER 11 : JACOBI DAVIDSON METHOD Heinrich Voss voss@tu-harburg.de Hamburg University of Technology Institute of Numerical Simulation

### SOLVING MESH EIGENPROBLEMS WITH MULTIGRID EFFICIENCY

SOLVING MESH EIGENPROBLEMS WITH MULTIGRID EFFICIENCY KLAUS NEYMEYR ABSTRACT. Multigrid techniques can successfully be applied to mesh eigenvalue problems for elliptic differential operators. They allow

### Solution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method

Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts

### A short course on: Preconditioned Krylov subspace methods. Yousef Saad University of Minnesota Dept. of Computer Science and Engineering

A short course on: Preconditioned Krylov subspace methods Yousef Saad University of Minnesota Dept. of Computer Science and Engineering Universite du Littoral, Jan 19-3, 25 Outline Part 1 Introd., discretization

### On the numerical solution of large-scale linear matrix equations. V. Simoncini

On the numerical solution of large-scale linear matrix equations V. Simoncini Dipartimento di Matematica, Università di Bologna (Italy) valeria.simoncini@unibo.it 1 Some matrix equations Sylvester matrix

### Department of Computer Science, University of Illinois at Urbana-Champaign

Department of Computer Science, University of Illinois at Urbana-Champaign Probing for Schur Complements and Preconditioning Generalized Saddle-Point Problems Eric de Sturler, sturler@cs.uiuc.edu, http://www-faculty.cs.uiuc.edu/~sturler

### Mathematics and Computer Science

Technical Report TR-2007-002 Block preconditioning for saddle point systems with indefinite (1,1) block by Michele Benzi, Jia Liu Mathematics and Computer Science EMORY UNIVERSITY International Journal

### DELFT UNIVERSITY OF TECHNOLOGY

DELFT UNIVERSITY OF TECHNOLOGY REPORT 14-1 Nested Krylov methods for shifted linear systems M. Baumann and M. B. van Gizen ISSN 1389-652 Reports of the Delft Institute of Applied Mathematics Delft 214

### Stopping criteria for iterations in finite element methods

Stopping criteria for iterations in finite element methods M. Arioli, D. Loghin and A. J. Wathen CERFACS Technical Report TR/PA/03/21 Abstract This work extends the results of Arioli [1], [2] on stopping

### Summary of Iterative Methods for Non-symmetric Linear Equations That Are Related to the Conjugate Gradient (CG) Method

Summary of Iterative Methods for Non-symmetric Linear Equations That Are Related to the Conjugate Gradient (CG) Method Leslie Foster 11-5-2012 We will discuss the FOM (full orthogonalization method), CG,

### Numerical Methods in Matrix Computations

Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices

### Computational Linear Algebra

Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 4: Iterative Methods PD

### The Lanczos and conjugate gradient algorithms

The Lanczos and conjugate gradient algorithms Gérard MEURANT October, 2008 1 The Lanczos algorithm 2 The Lanczos algorithm in finite precision 3 The nonsymmetric Lanczos algorithm 4 The Golub Kahan bidiagonalization

Notes for 2016-11-18 1 Conjugate gradients We now turn to the method of conjugate gradients (CG), perhaps the best known of the Krylov subspace solvers. The CG iteration can be characterized as the iteration

### 9.1 Preconditioned Krylov Subspace Methods

Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete

### DELFT UNIVERSITY OF TECHNOLOGY

DELFT UNIVERSITY OF TECHNOLOGY REPORT 15-05 Induced Dimension Reduction method for solving linear matrix equations R. Astudillo and M. B. van Gijzen ISSN 1389-6520 Reports of the Delft Institute of Applied

### AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline

### ABSTRACT. Professor G.W. Stewart

ABSTRACT Title of dissertation: Residual Arnoldi Methods : Theory, Package, and Experiments Che-Rung Lee, Doctor of Philosophy, 2007 Dissertation directed by: Professor G.W. Stewart Department of Computer

### Topics. The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems

Topics The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems What about non-spd systems? Methods requiring small history Methods requiring large history Summary of solvers 1 / 52 Conjugate

### 4.8 Arnoldi Iteration, Krylov Subspaces and GMRES

48 Arnoldi Iteration, Krylov Subspaces and GMRES We start with the problem of using a similarity transformation to convert an n n matrix A to upper Hessenberg form H, ie, A = QHQ, (30) with an appropriate

### Conjugate gradient method. Descent method. Conjugate search direction. Conjugate Gradient Algorithm (294)

Conjugate gradient method Descent method Hestenes, Stiefel 1952 For A N N SPD In exact arithmetic, solves in N steps In real arithmetic No guaranteed stopping Often converges in many fewer than N steps

### Linear algebra issues in Interior Point methods for bound-constrained least-squares problems

Linear algebra issues in Interior Point methods for bound-constrained least-squares problems Stefania Bellavia Dipartimento di Energetica S. Stecco Università degli Studi di Firenze Joint work with Jacek

### EXTENDED KRYLOV SUBSPACE FOR PARAMETER DEPENDENT SYSTEMS

EXTENDED KRYLOV SUBSPACE FOR PARAMETER DEPENDENT SYSTEMS V. SIMONCINI Abstract. The Extended Krylov Subspace has recently received considerable attention as a powerful tool for matrix function evaluations

### Henk van der Vorst. Abstract. We discuss a novel approach for the computation of a number of eigenvalues and eigenvectors

Subspace Iteration for Eigenproblems Henk van der Vorst Abstract We discuss a novel approach for the computation of a number of eigenvalues and eigenvectors of the standard eigenproblem Ax = x. Our method

### PROJECTED GMRES AND ITS VARIANTS

PROJECTED GMRES AND ITS VARIANTS Reinaldo Astudillo Brígida Molina rastudillo@kuaimare.ciens.ucv.ve bmolina@kuaimare.ciens.ucv.ve Centro de Cálculo Científico y Tecnológico (CCCT), Facultad de Ciencias,

### Alternative correction equations in the Jacobi-Davidson method. Mathematical Institute. Menno Genseberger and Gerard L. G.

Universiteit-Utrecht * Mathematical Institute Alternative correction equations in the Jacobi-Davidson method by Menno Genseberger and Gerard L. G. Sleijpen Preprint No. 173 June 1998, revised: March 1999

### The quadratic eigenvalue problem (QEP) is to find scalars λ and nonzero vectors u satisfying

I.2 Quadratic Eigenvalue Problems 1 Introduction The quadratic eigenvalue problem QEP is to find scalars λ and nonzero vectors u satisfying where Qλx = 0, 1.1 Qλ = λ 2 M + λd + K, M, D and K are given

### Eigenvalue Problems Computation and Applications

Eigenvalue ProblemsComputation and Applications p. 1/36 Eigenvalue Problems Computation and Applications Che-Rung Lee cherung@gmail.com National Tsing Hua University Eigenvalue ProblemsComputation and

### Computational Linear Algebra

Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD

### ECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems. Part I: Review of basic theory of eigenvalue problems

ECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems Part I: Review of basic theory of eigenvalue problems 1. Let A C n n. (a) A scalar λ is an eigenvalue of an n n A

### A DISSERTATION. Extensions of the Conjugate Residual Method. by Tomohiro Sogabe. Presented to

A DISSERTATION Extensions of the Conjugate Residual Method ( ) by Tomohiro Sogabe Presented to Department of Applied Physics, The University of Tokyo Contents 1 Introduction 1 2 Krylov subspace methods

### A Jacobi Davidson type method for the generalized singular value problem

A Jacobi Davidson type method for the generalized singular value problem M. E. Hochstenbach a, a Department of Mathematics and Computing Science, Eindhoven University of Technology, P.O. Box 513, 5600

### MICHIEL E. HOCHSTENBACH

VARIATIONS ON HARMONIC RAYLEIGH RITZ FOR STANDARD AND GENERALIZED EIGENPROBLEMS MICHIEL E. HOCHSTENBACH Abstract. We present several variations on the harmonic Rayleigh Ritz method. First, we introduce

### Harmonic Projection Methods for Large Non-symmetric Eigenvalue Problems

NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS Numer. Linear Algebra Appl., 5, 33 55 (1998) Harmonic Projection Methods for Large Non-symmetric Eigenvalue Problems Ronald B. Morgan 1 and Min Zeng 2 1 Department

### Some minimization problems

Week 13: Wednesday, Nov 14 Some minimization problems Last time, we sketched the following two-step strategy for approximating the solution to linear systems via Krylov subspaces: 1. Build a sequence of

### Algorithms that use the Arnoldi Basis

AMSC 600 /CMSC 760 Advanced Linear Numerical Analysis Fall 2007 Arnoldi Methods Dianne P. O Leary c 2006, 2007 Algorithms that use the Arnoldi Basis Reference: Chapter 6 of Saad The Arnoldi Basis How to

### STEEPEST DESCENT AND CONJUGATE GRADIENT METHODS WITH VARIABLE PRECONDITIONING

SIAM J. MATRIX ANAL. APPL. Vol.?, No.?, pp.?? c 2007 Society for Industrial and Applied Mathematics STEEPEST DESCENT AND CONJUGATE GRADIENT METHODS WITH VARIABLE PRECONDITIONING ANDREW V. KNYAZEV AND ILYA

### A comparison of solvers for quadratic eigenvalue problems from combustion

INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN FLUIDS [Version: 2002/09/18 v1.01] A comparison of solvers for quadratic eigenvalue problems from combustion C. Sensiau 1, F. Nicoud 2,, M. van Gijzen 3,

### Contents. Preface... xi. Introduction...

Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures... 1 1.1. Different types of parallelism... 1 1.1.1. Overlap, concurrency and parallelism... 1 1.1.2. Temporal and spatial parallelism

### DELFT UNIVERSITY OF TECHNOLOGY

DELFT UNIVERSITY OF TECHNOLOGY REPORT -09 Computational and Sensitivity Aspects of Eigenvalue-Based Methods for the Large-Scale Trust-Region Subproblem Marielba Rojas, Bjørn H. Fotland, and Trond Steihaug