GMRES: Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
|
|
- Caroline Logan
- 5 years ago
- Views:
Transcription
1 GMRES: Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University December 4, 2011 T.-M. Huang (NTNU) GMRES December 4, / 26
2 Ref: SISC, 1984, Saad Theorem 1 (Implicit Q theorem) Let AV 1 = V 1 H 1 and AV 2 = V 2 H 2, where H 1, H 2 are Hessenberg and V 1, V 2 are unitary with V 1 e 1 = V 2 e 1 = q 1. Then V 1 = V 2 and H 1 = H 2. Proof A [ ] [ ] v 1 v 2 v n = v1 v 2 v n h 11 h 12 h 1n h 21 h 22 h 2n h n,n 1 h nn with v T i v j = δ ij, i, j = 1,..., n T.-M. Huang (NTNU) GMRES December 4, / 26
3 Arnoldi Algorithm Input: Given v 1 with v 1 2 = 1; Output: Arnoldi factorization: AV k = V k H k + h k+1,k v k+1 e T k. 1: Set k = 0. 2: repeat 3: Compute h ik = (Av k, v i ) for i = 1, 2,..., k; 4: Compute ṽ k+1 = Av k k i=1 h ikv i ; 5: Compute h k+1,k = ṽ k+1 2 ; 6: Compute v k+1 = ṽ k+1 /h k+1,k ; 7: Set k = k + 1; 8: until convergent T.-M. Huang (NTNU) GMRES December 4, / 26
4 Remark 1 (a) Let V k = [v 1,, v k ] R n k where v j, for j = 1,..., k, is generated by Arnoldi algorithm. Then H k V T k AV k is upper k k Hessenberg. (b) Arnoldi s original method was a Galerkin method for approximate the eigenvalue of A by H k. T.-M. Huang (NTNU) GMRES December 4, / 26
5 In order to solve Ax = b by the Galerkin method using < K k > < V k >, we seek an approximate solution x k = x 0 + z k with and r 0 = b Ax 0. Definition 2 z k K k =< r 0, Ar 0,, A k 1 r 0 > {x k } is said to be satisfied the Galerkin condition if r k b Ax k is orthogonal to K k for each k. The Galerkin method can be stated as that find such that x k = x 0 + z k with z k V k (1) (b Ax k, v) = 0, v V k, T.-M. Huang (NTNU) GMRES December 4, / 26
6 which is equivalent to find z k V k y k V k (2) such that Substituting (2) into (3), we get which implies that (r 0 Az k, v) = 0, v V k. (3) V T k (r 0 AV k y k ) = 0, y k = (V T k AV k) 1 r 0 e 1. (4) T.-M. Huang (NTNU) GMRES December 4, / 26
7 Since V k is computed by the Arnoldi algorithm with v 1 = r 0 / r 0, y k in (4) can be represented as y k = H 1 k r 0 e 1. Substituting it into (2) and (1), we get x k = x 0 + V k H 1 k r 0 e 1. Using the result that AV k = V k H k + h k+1,k v k+1 e T k, r k can be reformulated as r k = b Ax k = r 0 AV k y k = r 0 (V k H k + h k+1,k v k+1 e T k )y k = r 0 V k r 0 e 1 h k+1,k e T k y kv k+1 = (h k+1,k e T k y k)v k+1. T.-M. Huang (NTNU) GMRES December 4, / 26
8 The generalized minimal residual (GMRES) algorithm The approximate solution of the form x 0 + z k, which minimizes the residual norm over z k K k, can in principle be obtained by following algorithms: The ORTHODIR algorithm of Jea and Young; the generalized conjugate residual method (GCR); GMRES. Let h 1,1 h 1,2 h 1,k h 2,1 h 2,2 h 2,k V k = [v 1,, v k ], Hk = R (k+1) k..... hk,k 1 h k,k 0 0 h k+1,k T.-M. Huang (NTNU) GMRES December 4, / 26
9 By Arnoldi algorithm, we have To solve the least square problem: AV k = V k+1 Hk. (5) min z K k r o Az 2 = min z K k b A(x o + z) 2, (6) where K k =< r o, Ar o,, A k 1 r o >=< v 1,, v k > with v 1 = ro r o 2. T.-M. Huang (NTNU) GMRES December 4, / 26
10 Set z = V k y, the least square problem (6) is equivalent to min J(y) = min βv 1 AV k y 2, β = r o 2. (7) y Rk y R k Using (5), we have J(y) = V k+1 (βe 1 H ) k y 2 = βe 1 H k y 2. (8) Hence, the solution of the least square (6) is x k = x o + V k y k, where y k minimize the function J(y) defined by (8) over y R k. T.-M. Huang (NTNU) GMRES December 4, / 26
11 GMRES Algorithm Input: Choose x 0, compute r 0 = b Ax 0 and v 1 = r 0 / r 0 ; Output: Solution of linear system Ax = b. 1: for j = 1, 2,..., k do 2: Compute h ij = (Av j, v i ) for i = 1, 2,..., j; 3: Compute ṽ j+1 = Av j j i=1 h ijv i ; 4: Compute h j+1,j = ṽ j+1 2 ; 5: Compute v j+1 = ṽ j+1 /h j+1,j ; 6: end for 7: Form the solution: x k = x 0 + V k y k, where y k minimizes J(y) in (8). Difficulties: when k is increasing, storage for v j, like k, the number of multiplications is like 1 2 k2 N. T.-M. Huang (NTNU) GMRES December 4, / 26
12 GMRES(m) Algorithm Input: Choose x 0, compute r 0 = b Ax 0 and v 1 = r 0 / r 0 ; Output: Solution of linear system Ax = b. 1: for j = 1, 2,..., m do 2: Compute h ij = (Av j, v i ) for i = 1, 2,..., j; 3: Compute ṽ j+1 = Av j j i=1 h ijv i ; 4: Compute h j+1,j = ṽ j+1 2 ; 5: Compute v j+1 = ṽ j+1 /h j+1,j ; 6: end for 7: Form the solution: x m = x 0 + V m y m, where y m minimizes βe 1 H m y for y R m. 8: Restart: Compute r m = b Ax m ; 9: if r m is small, then 10: stop, 11: else 12: Compute x 0 = x m and v 1 = r m / r m, GoTo for step. 13: end if T.-M. Huang (NTNU) GMRES December 4, / 26
13 Practical Implementation: Consider QR factorization of H k Consider the matrix H k. We want to solve the least squares problem: min βe 1 H k y 2. y R k Assume Givens rotations F i, i = 1,..., j such that F j F 1 Hj =F j F = R j R (j+1) j. 0 Definition of Givens T.-M. Huang (NTNU) GMRES December 4, / 26
14 In order to obtain R j+1 we must start by premultiptying the new column by the previous rotations. H j+1 = F j F 1 Hj+1 = r 0 h The principal upper (j + 1) j submatrix is nothing but R j, and h := h j+2,j+1 is not affected by the previous rotations. The next rotation F j+1 defined by { cj+1 r/(r 2 + h 2 ) 1/2, s j+1 = h/(r 2 + h 2 ) 1/2. T.-M. Huang (NTNU) GMRES December 4, / 26
15 Thus, after k steps of the above process, we have achieved Q k Hk = R k where Q k is a (k + 1) (k + 1) unitary matrix and J(y) = βe 1 H k y = Q k (βe 1 H ) k y = g k R k y, (9) where g k Q k βe 1. Since the last row of R k is a zero row, the 1 minimization of (9) is achieved at y k = R k g k, where R k and g k are removed the last row of R k and the last component of g k, respectively. Proposition 1 r k = b Ax k = The (k+1)-st component of g k. T.-M. Huang (NTNU) GMRES December 4, / 26
16 Proposition 2 The solution x j produced by GMRES at step j is exact which is equivalent to (i) The algorithm breaks down at step j, (ii) ṽ j+1 = 0, (iii) h j+1,j = 0, (iv) The degree of the minimal polynomial of r 0 is j. Corollary 3 For an n n problem GMRES terminates at most n steps. This uncommon type of breakdown is sometimes referred to as a Lucky breakdown is the context of the Lanczos algorithm. T.-M. Huang (NTNU) GMRES December 4, / 26
17 Proposition 3 Suppose that A is diagonalizable so that A = XDX 1 and let Then ε (m) = where κ(x) = X X 1. min p P m,p(0)=1 max p(λ i). λ i σ(a) r m+1 κ(x)ε (m) r 0, When A is positive real with symmetric part M, it holds that r m [1 α/β] m/2 r 0, where α = (λ min (M)) 2 and β = λ max (A T A). This proves the convergence of GMRES(m) for all m, when A is positive real. T.-M. Huang (NTNU) GMRES December 4, / 26
18 Theorem 4 Assume λ 1,..., λ ν of A with positive(negative) real parts and the other eigenvalues enclosed in a circle centered at C with C > 0 and have radius R with C > R. Then where ε (m) D = [ ] R m ν max C j=ν+1,,n ν i=1 λ i λ j λ i [ D d ] 2 [ ] R m ν C max i=1,,ν λ i λ j and d = min i. i=1,,ν j=ν+1,,n T.-M. Huang (NTNU) GMRES December 4, / 26
19 Proof. Consider p(z) = r(z)q(z) where r(z) = (1 z/λ 1 ) (1 z/λ ν ) and q(z) arbitrary polynomial of deg m ν such that q(0) = 1. Since p(0) = 1 and p(λ i ) = 0, for i = 1,..., ν, we have ε (m) It is easily seen that max p(λ j) j=ν+1,,n max r(λ j) max q(λ j). j=ν+1,,n j=ν+1,,n max r(λ j) = j=ν+1,,n max j=ν+1,,n ν i=1 λ i λ j λ i [ ] D ν. d By maximum principle, the maximum of q(z) for z {λ j } N j=ν+1 is on the circle. Taking σ(z) = [(C z)/c] m ν whose maximum on the circle is (R/C) m ν yields the desired result. T.-M. Huang (NTNU) GMRES December 4, / 26
20 Corollary 5 Under the assumptions of Proposition 3 and Theorem 4, GMRES(m) converges for any initial x 0 if [ ] / [ DC C m > νlog dr κ(x)1/ν Log R. T.-M. Huang (NTNU) GMRES December 4, / 26
21 Appendix Proof of Implicit Q Theorem Let A[q 1 q 2 q n] = [q 1 q 2 q n] h 11 h 12 h 1n. h 21 h hn 1,n 0 0 h n,n 1 h nn. (10) T.-M. Huang (NTNU) GMRES December 4, / 26
22 Then we have Aq 1 = h 11 q 1 + h 21 q 2. (11) Since q 1 q 2, it implies that h 11 = q1aq 1 /q1q 1. From (11), we get that q 2 h 21 q 2 = Aq 1 h 11 q 1. That is q 2 = q 2 / q 2 2 and h 21 = q 2 2. T.-M. Huang (NTNU) GMRES December 4, / 26
23 Similarly, from (10), Aq 2 = h 12 q 1 + h 22 q 2 + h 32 q 3, where h 12 = q 1Aq 2 and h 22 = q 2Aq 2. Let q 3 = Aq 2 h 12 q 1 + h 22 q 2. Then q 3 = q 3 / q 3 2 and h 32 = q 3, and so on. T.-M. Huang (NTNU) GMRES December 4, / 26
24 Therefore, [q 1,, q n ] are uniquely determined by q 1. Thus, uniqueness holds. Let K n = [v 1, Av 1,, A n 1 v 1 ] with v 1 2 = 1 is nonsingular. K n = U n R n and U n e 1 = v 1. Then AK n = K n C n = [v 1, Av 1,, A n 1 v 1 ] (12) T.-M. Huang (NTNU) GMRES December 4, / 26
25 Since K n is nonsingular, (12) implies that That is A = K n C n K 1 n = (U n R n )C n (Rn 1 Un 1 ). AU n = U n (R n C n R 1 n ), where (R n C n Rn 1 ) is Hessenberg and U n e 1 = v 1. Because < U n >=< K n >, find AV n = V n H n by any method with V n e 1 = v 1, then it holds that V n = U n, i.e., v (i) n = u (i) n for i = 1,, n. Back to Theorem T.-M. Huang (NTNU) GMRES December 4, / 26
26 Definition 6 (Givens rotation) A plane rotation (also called a Givens rotation) is a matrix of the form [ ] c s G = s c where c 2 + s 2 = 1. Given a 0 and b, set v = a 2 + b 2, c = a /v and s = a a b v, then [ c s s c ] [ a b ] = [ v a a 0 ]. Back to Practice GMRES T.-M. Huang (NTNU) GMRES December 4, / 26
4.8 Arnoldi Iteration, Krylov Subspaces and GMRES
48 Arnoldi Iteration, Krylov Subspaces and GMRES We start with the problem of using a similarity transformation to convert an n n matrix A to upper Hessenberg form H, ie, A = QHQ, (30) with an appropriate
More informationKrylov Subspace Methods that Are Based on the Minimization of the Residual
Chapter 5 Krylov Subspace Methods that Are Based on the Minimization of the Residual Remark 51 Goal he goal of these methods consists in determining x k x 0 +K k r 0,A such that the corresponding Euclidean
More informationA short course on: Preconditioned Krylov subspace methods. Yousef Saad University of Minnesota Dept. of Computer Science and Engineering
A short course on: Preconditioned Krylov subspace methods Yousef Saad University of Minnesota Dept. of Computer Science and Engineering Universite du Littoral, Jan 19-3, 25 Outline Part 1 Introd., discretization
More informationFEM and sparse linear system solving
FEM & sparse linear system solving, Lecture 9, Nov 19, 2017 1/36 Lecture 9, Nov 17, 2017: Krylov space methods http://people.inf.ethz.ch/arbenz/fem17 Peter Arbenz Computer Science Department, ETH Zürich
More informationAlgorithms that use the Arnoldi Basis
AMSC 600 /CMSC 760 Advanced Linear Numerical Analysis Fall 2007 Arnoldi Methods Dianne P. O Leary c 2006, 2007 Algorithms that use the Arnoldi Basis Reference: Chapter 6 of Saad The Arnoldi Basis How to
More informationKrylov subspace projection methods
I.1.(a) Krylov subspace projection methods Orthogonal projection technique : framework Let A be an n n complex matrix and K be an m-dimensional subspace of C n. An orthogonal projection technique seeks
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 9 Minimizing Residual CG
More informationThe Lanczos and conjugate gradient algorithms
The Lanczos and conjugate gradient algorithms Gérard MEURANT October, 2008 1 The Lanczos algorithm 2 The Lanczos algorithm in finite precision 3 The nonsymmetric Lanczos algorithm 4 The Golub Kahan bidiagonalization
More informationKrylov Space Methods. Nonstationary sounds good. Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 1 / 17
Krylov Space Methods Nonstationary sounds good Radu Trîmbiţaş Babeş-Bolyai University Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 1 / 17 Introduction These methods are used both to solve
More informationSummary of Iterative Methods for Non-symmetric Linear Equations That Are Related to the Conjugate Gradient (CG) Method
Summary of Iterative Methods for Non-symmetric Linear Equations That Are Related to the Conjugate Gradient (CG) Method Leslie Foster 11-5-2012 We will discuss the FOM (full orthogonalization method), CG,
More informationKrylov Subspace Methods for Large/Sparse Eigenvalue Problems
Krylov Subspace Methods for Large/Sparse Eigenvalue Problems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan April 17, 2012 T.-M. Huang (Taiwan Normal University) Krylov
More informationOn prescribing Ritz values and GMRES residual norms generated by Arnoldi processes
On prescribing Ritz values and GMRES residual norms generated by Arnoldi processes Jurjen Duintjer Tebbens Institute of Computer Science Academy of Sciences of the Czech Republic joint work with Gérard
More informationOn the influence of eigenvalues on Bi-CG residual norms
On the influence of eigenvalues on Bi-CG residual norms Jurjen Duintjer Tebbens Institute of Computer Science Academy of Sciences of the Czech Republic duintjertebbens@cs.cas.cz Gérard Meurant 30, rue
More informationThe quadratic eigenvalue problem (QEP) is to find scalars λ and nonzero vectors u satisfying
I.2 Quadratic Eigenvalue Problems 1 Introduction The quadratic eigenvalue problem QEP is to find scalars λ and nonzero vectors u satisfying where Qλx = 0, 1.1 Qλ = λ 2 M + λd + K, M, D and K are given
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More informationIDR(s) as a projection method
Delft University of Technology Faculty of Electrical Engineering, Mathematics and Computer Science Delft Institute of Applied Mathematics IDR(s) as a projection method A thesis submitted to the Delft Institute
More informationIterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)
Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential
More informationSolution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method
Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts
More informationA stable variant of Simpler GMRES and GCR
A stable variant of Simpler GMRES and GCR Miroslav Rozložník joint work with Pavel Jiránek and Martin H. Gutknecht Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic miro@cs.cas.cz,
More informationLast Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection
Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue
More informationPROJECTED GMRES AND ITS VARIANTS
PROJECTED GMRES AND ITS VARIANTS Reinaldo Astudillo Brígida Molina rastudillo@kuaimare.ciens.ucv.ve bmolina@kuaimare.ciens.ucv.ve Centro de Cálculo Científico y Tecnológico (CCCT), Facultad de Ciencias,
More informationIterative Methods for Sparse Linear Systems
Iterative Methods for Sparse Linear Systems Luca Bergamaschi e-mail: berga@dmsa.unipd.it - http://www.dmsa.unipd.it/ berga Department of Mathematical Methods and Models for Scientific Applications University
More informationThe German word eigen is cognate with the Old English word āgen, which became owen in Middle English and own in modern English.
Chapter 4 EIGENVALUE PROBLEM The German word eigen is cognate with the Old English word āgen, which became owen in Middle English and own in modern English. 4.1 Mathematics 4.2 Reduction to Upper Hessenberg
More informationSolution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method
Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts
More informationCharles University Faculty of Mathematics and Physics DOCTORAL THESIS. Krylov subspace approximations in linear algebraic problems
Charles University Faculty of Mathematics and Physics DOCTORAL THESIS Iveta Hnětynková Krylov subspace approximations in linear algebraic problems Department of Numerical Mathematics Supervisor: Doc. RNDr.
More informationIterative methods for Linear System
Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and
More informationResearch Article Some Generalizations and Modifications of Iterative Methods for Solving Large Sparse Symmetric Indefinite Linear Systems
Abstract and Applied Analysis Article ID 237808 pages http://dxdoiorg/055/204/237808 Research Article Some Generalizations and Modifications of Iterative Methods for Solving Large Sparse Symmetric Indefinite
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 4: Iterative Methods PD
More informationON ORTHOGONAL REDUCTION TO HESSENBERG FORM WITH SMALL BANDWIDTH
ON ORTHOGONAL REDUCTION TO HESSENBERG FORM WITH SMALL BANDWIDTH V. FABER, J. LIESEN, AND P. TICHÝ Abstract. Numerous algorithms in numerical linear algebra are based on the reduction of a given matrix
More informationc 2005 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 26, No. 4, pp. 1001 1021 c 2005 Society for Industrial and Applied Mathematics BREAKDOWN-FREE GMRES FOR SINGULAR SYSTEMS LOTHAR REICHEL AND QIANG YE Abstract. GMRES is a
More informationMatrices, Moments and Quadrature, cont d
Jim Lambers CME 335 Spring Quarter 2010-11 Lecture 4 Notes Matrices, Moments and Quadrature, cont d Estimation of the Regularization Parameter Consider the least squares problem of finding x such that
More informationON THE GLOBAL KRYLOV SUBSPACE METHODS FOR SOLVING GENERAL COUPLED MATRIX EQUATIONS
ON THE GLOBAL KRYLOV SUBSPACE METHODS FOR SOLVING GENERAL COUPLED MATRIX EQUATIONS Fatemeh Panjeh Ali Beik and Davod Khojasteh Salkuyeh, Department of Mathematics, Vali-e-Asr University of Rafsanjan, Rafsanjan,
More informationLARGE SPARSE EIGENVALUE PROBLEMS. General Tools for Solving Large Eigen-Problems
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization General Tools for Solving Large Eigen-Problems
More informationA DISSERTATION. Extensions of the Conjugate Residual Method. by Tomohiro Sogabe. Presented to
A DISSERTATION Extensions of the Conjugate Residual Method ( ) by Tomohiro Sogabe Presented to Department of Applied Physics, The University of Tokyo Contents 1 Introduction 1 2 Krylov subspace methods
More informationIterative techniques in matrix algebra
Iterative techniques in matrix algebra Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Norms of vectors and matrices 2 Eigenvalues and
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 19: More on Arnoldi Iteration; Lanczos Iteration Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 17 Outline 1
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationInstitute for Advanced Computer Studies. Department of Computer Science. Iterative methods for solving Ax = b. GMRES/FOM versus QMR/BiCG
University of Maryland Institute for Advanced Computer Studies Department of Computer Science College Park TR{96{2 TR{3587 Iterative methods for solving Ax = b GMRES/FOM versus QMR/BiCG Jane K. Cullum
More informationLARGE SPARSE EIGENVALUE PROBLEMS
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization 14-1 General Tools for Solving Large Eigen-Problems
More informationMath 577 Assignment 7
Math 577 Assignment 7 Thanks for Yu Cao 1. Solution. The linear system being solved is Ax = 0, where A is a (n 1 (n 1 matrix such that 2 1 1 2 1 A =......... 1 2 1 1 2 and x = (U 1, U 2,, U n 1. By the
More informationMath 504 (Fall 2011) 1. (*) Consider the matrices
Math 504 (Fall 2011) Instructor: Emre Mengi Study Guide for Weeks 11-14 This homework concerns the following topics. Basic definitions and facts about eigenvalues and eigenvectors (Trefethen&Bau, Lecture
More informationIntroduction to Iterative Solvers of Linear Systems
Introduction to Iterative Solvers of Linear Systems SFB Training Event January 2012 Prof. Dr. Andreas Frommer Typeset by Lukas Krämer, Simon-Wolfgang Mages and Rudolf Rödl 1 Classes of Matrices and their
More informationCourse Notes: Week 1
Course Notes: Week 1 Math 270C: Applied Numerical Linear Algebra 1 Lecture 1: Introduction (3/28/11) We will focus on iterative methods for solving linear systems of equations (and some discussion of eigenvalues
More informationTsung-Ming Huang. Matrix Computation, 2016, NTNU
Tsung-Ming Huang Matrix Computation, 2016, NTNU 1 Plan Gradient method Conjugate gradient method Preconditioner 2 Gradient method 3 Theorem Ax = b, A : s.p.d Definition A : symmetric positive definite
More informationEigenvalue Problems. Eigenvalue problems occur in many areas of science and engineering, such as structural analysis
Eigenvalue Problems Eigenvalue problems occur in many areas of science and engineering, such as structural analysis Eigenvalues also important in analyzing numerical methods Theory and algorithms apply
More informationConjugate Gradient Method
Conjugate Gradient Method Tsung-Ming Huang Department of Mathematics National Taiwan Normal University October 10, 2011 T.M. Huang (NTNU) Conjugate Gradient Method October 10, 2011 1 / 36 Outline 1 Steepest
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 4 Eigenvalue Problems Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationEigenvalues and Eigenvectors
Chapter 1 Eigenvalues and Eigenvectors Among problems in numerical linear algebra, the determination of the eigenvalues and eigenvectors of matrices is second in importance only to the solution of linear
More informationMath 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm
Math 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm References: Trefethen & Bau textbook Eigenvalue problem: given a matrix A, find
More information5.3 The Power Method Approximation of the Eigenvalue of Largest Module
192 5 Approximation of Eigenvalues and Eigenvectors 5.3 The Power Method The power method is very good at approximating the extremal eigenvalues of the matrix, that is, the eigenvalues having largest and
More informationEigenvalue and Eigenvector Problems
Eigenvalue and Eigenvector Problems An attempt to introduce eigenproblems Radu Trîmbiţaş Babeş-Bolyai University April 8, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) Eigenvalue and Eigenvector Problems
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationDELFT UNIVERSITY OF TECHNOLOGY
DELFT UNIVERSITY OF TECHNOLOGY REPORT 06-05 Solution of the incompressible Navier Stokes equations with preconditioned Krylov subspace methods M. ur Rehman, C. Vuik G. Segal ISSN 1389-6520 Reports of the
More information1 Extrapolation: A Hint of Things to Come
Notes for 2017-03-24 1 Extrapolation: A Hint of Things to Come Stationary iterations are simple. Methods like Jacobi or Gauss-Seidel are easy to program, and it s (relatively) easy to analyze their convergence.
More information13-2 Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2
The QR algorithm The most common method for solving small (dense) eigenvalue problems. The basic algorithm: QR without shifts 1. Until Convergence Do: 2. Compute the QR factorization A = QR 3. Set A :=
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationLecture 17 Methods for System of Linear Equations: Part 2. Songting Luo. Department of Mathematics Iowa State University
Lecture 17 Methods for System of Linear Equations: Part 2 Songting Luo Department of Mathematics Iowa State University MATH 481 Numerical Methods for Differential Equations Songting Luo ( Department of
More information6.4 Krylov Subspaces and Conjugate Gradients
6.4 Krylov Subspaces and Conjugate Gradients Our original equation is Ax = b. The preconditioned equation is P Ax = P b. When we write P, we never intend that an inverse will be explicitly computed. P
More informationSOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA
1 SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA 2 OUTLINE Sparse matrix storage format Basic factorization
More informationETNA Kent State University
Electronic Transactions on Numerical Analysis. Volume 33, pp. 63-83, 2009. Copyright 2009,. ISSN 1068-9613. ETNA SIMPLE SQUARE SMOOTHING REGULARIZATION OPERATORS LOTHAR REICHEL AND QIANG YE Dedicated to
More informationMath Matrix Algebra
Math 44 - Matrix Algebra Review notes - (Alberto Bressan, Spring 7) sec: Orthogonal diagonalization of symmetric matrices When we seek to diagonalize a general n n matrix A, two difficulties may arise:
More informationAugmented GMRES-type methods
Augmented GMRES-type methods James Baglama 1 and Lothar Reichel 2, 1 Department of Mathematics, University of Rhode Island, Kingston, RI 02881. E-mail: jbaglama@math.uri.edu. Home page: http://hypatia.math.uri.edu/
More informationSolving large scale eigenvalue problems
arge scale eigenvalue problems, Lecture 4, March 14, 2018 1/41 Lecture 4, March 14, 2018: The QR algorithm http://people.inf.ethz.ch/arbenz/ewp/ Peter Arbenz Computer Science Department, ETH Zürich E-mail:
More informationDELFT UNIVERSITY OF TECHNOLOGY
DELFT UNIVERSITY OF TECHNOLOGY REPORT 10-14 On the Convergence of GMRES with Invariant-Subspace Deflation M.C. Yeung, J.M. Tang, and C. Vuik ISSN 1389-6520 Reports of the Delft Institute of Applied Mathematics
More informationChapter 7. Iterative methods for large sparse linear systems. 7.1 Sparse matrix algebra. Large sparse matrices
Chapter 7 Iterative methods for large sparse linear systems In this chapter we revisit the problem of solving linear systems of equations, but now in the context of large sparse systems. The price to pay
More informationABSTRACT OF DISSERTATION. Wei Zhang. The Graduate School. University of Kentucky
ABSTRACT OF DISSERTATION Wei Zhang The Graduate School University of Kentucky 2007 GMRES ON A TRIDIAGONAL TOEPLITZ LINEAR SYSTEM ABSTRACT OF DISSERTATION A dissertation submitted in partial fulfillment
More informationSolving large scale eigenvalue problems
arge scale eigenvalue problems, Lecture 5, March 23, 2016 1/30 Lecture 5, March 23, 2016: The QR algorithm II http://people.inf.ethz.ch/arbenz/ewp/ Peter Arbenz Computer Science Department, ETH Zürich
More informationGaussian Elimination for Linear Systems
Gaussian Elimination for Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University October 3, 2011 1/56 Outline 1 Elementary matrices 2 LR-factorization 3 Gaussian elimination
More informationModel reduction of large-scale dynamical systems
Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation Thanos Antoulas Rice University and Jacobs University email: aca@rice.edu URL: www.ece.rice.edu/
More informationITERATIVE METHODS BASED ON KRYLOV SUBSPACES
ITERATIVE METHODS BASED ON KRYLOV SUBSPACES LONG CHEN We shall present iterative methods for solving linear algebraic equation Au = b based on Krylov subspaces We derive conjugate gradient (CG) method
More informationM.A. Botchev. September 5, 2014
Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev
More informationInexactness and flexibility in linear Krylov solvers
Inexactness and flexibility in linear Krylov solvers Luc Giraud ENSEEIHT (N7) - IRIT, Toulouse Matrix Analysis and Applications CIRM Luminy - October 15-19, 2007 in honor of Gérard Meurant for his 60 th
More informationThe rate of convergence of the GMRES method
The rate of convergence of the GMRES method Report 90-77 C. Vuik Technische Universiteit Delft Delft University of Technology Faculteit der Technische Wiskunde en Informatica Faculty of Technical Mathematics
More informationProjection-based iterative methods. p. 1/49
Projection-based iterative methods p. 1/49 p. 2/49 TDB NLA Parallel Algorithms for Scientific Computing Shortly on projectors p. 3/49 Projectors and properties Definitions: Consider C n and a mapping P
More informationStructured Krylov Subspace Methods for Eigenproblems with Spectral Symmetries
Structured Krylov Subspace Methods for Eigenproblems with Spectral Symmetries Fakultät für Mathematik TU Chemnitz, Germany Peter Benner benner@mathematik.tu-chemnitz.de joint work with Heike Faßbender
More informationThe amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A.
AMSC/CMSC 661 Scientific Computing II Spring 2005 Solution of Sparse Linear Systems Part 2: Iterative methods Dianne P. O Leary c 2005 Solving Sparse Linear Systems: Iterative methods The plan: Iterative
More informationLab 1: Iterative Methods for Solving Linear Systems
Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as
More informationANALYSIS OF SOME KRYLOV SUBSPACE APPROXIMATIONS TO THE MATRIX EXPONENTIAL OPERATOR
ANALYSIS OF SOME KRYLOV SUBSPACE APPROXIMATIONS TO THE MATRIX EXPONENTIAL OPERATOR Y. SAAD Abstract. In this note we present a theoretical analysis of some Krylov subspace approximations to the matrix
More informationNumerical Methods for Solving Large Scale Eigenvalue Problems
Peter Arbenz Computer Science Department, ETH Zürich E-mail: arbenz@inf.ethz.ch arge scale eigenvalue problems, Lecture 2, February 28, 2018 1/46 Numerical Methods for Solving Large Scale Eigenvalue Problems
More informationECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems. Part I: Review of basic theory of eigenvalue problems
ECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems Part I: Review of basic theory of eigenvalue problems 1. Let A C n n. (a) A scalar λ is an eigenvalue of an n n A
More informationRecent computational developments in Krylov Subspace Methods for linear systems. Valeria Simoncini and Daniel B. Szyld
Recent computational developments in Krylov Subspace Methods for linear systems Valeria Simoncini and Daniel B. Szyld A later version appeared in : Numerical Linear Algebra w/appl., 2007, v. 14(1), pp.1-59.
More informationConjugate gradient method. Descent method. Conjugate search direction. Conjugate Gradient Algorithm (294)
Conjugate gradient method Descent method Hestenes, Stiefel 1952 For A N N SPD In exact arithmetic, solves in N steps In real arithmetic No guaranteed stopping Often converges in many fewer than N steps
More informationABSTRACT OF DISSERTATION. Ping Zhang
ABSTRACT OF DISSERTATION Ping Zhang The Graduate School University of Kentucky 2009 Iterative Methods for Computing Eigenvalues and Exponentials of Large Matrices ABSTRACT OF DISSERTATION A dissertation
More informationEIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4
EIGENVALUE PROBLEMS EIGENVALUE PROBLEMS p. 1/4 EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue
More informationApplied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.4: Krylov Subspace Methods 2 / 51 Krylov Subspace Methods In this chapter we give
More informationLecture notes: Applied linear algebra Part 2. Version 1
Lecture notes: Applied linear algebra Part 2. Version 1 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 First, some exercises: xercise 0.1 (2 Points) Another least
More informationApplied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.2: Fundamentals 2 / 31 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors of
More informationNumerical Solution of Linear Eigenvalue Problems
Numerical Solution of Linear Eigenvalue Problems Jessica Bosch and Chen Greif Abstract We review numerical methods for computing eigenvalues of matrices We start by considering the computation of the dominant
More informationMath 408 Advanced Linear Algebra
Math 408 Advanced Linear Algebra Chi-Kwong Li Chapter 4 Hermitian and symmetric matrices Basic properties Theorem Let A M n. The following are equivalent. Remark (a) A is Hermitian, i.e., A = A. (b) x
More informationMath Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 2: Direct Methods PD Dr.
More informationEigenvalue Problems and Singular Value Decomposition
Eigenvalue Problems and Singular Value Decomposition Sanzheng Qiao Department of Computing and Software McMaster University August, 2012 Outline 1 Eigenvalue Problems 2 Singular Value Decomposition 3 Software
More informationMATH 511 ADVANCED LINEAR ALGEBRA SPRING 2006
MATH 511 ADVANCED LINEAR ALGEBRA SPRING 2006 Sherod Eubanks HOMEWORK 2 2.1 : 2, 5, 9, 12 2.3 : 3, 6 2.4 : 2, 4, 5, 9, 11 Section 2.1: Unitary Matrices Problem 2 If λ σ(u) and U M n is unitary, show that
More informationANY FINITE CONVERGENCE CURVE IS POSSIBLE IN THE INITIAL ITERATIONS OF RESTARTED FOM
Electronic Transactions on Numerical Analysis. Volume 45, pp. 133 145, 2016. Copyright c 2016,. ISSN 1068 9613. ETNA ANY FINITE CONVERGENCE CURVE IS POSSIBLE IN THE INITIAL ITERATIONS OF RESTARTED FOM
More informationEIGENVALUE PROBLEMS (EVP)
EIGENVALUE PROBLEMS (EVP) (Golub & Van Loan: Chaps 7-8; Watkins: Chaps 5-7) X.-W Chang and C. C. Paige PART I. EVP THEORY EIGENVALUES AND EIGENVECTORS Let A C n n. Suppose Ax = λx with x 0, then x is a
More informationTopics. The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems
Topics The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems What about non-spd systems? Methods requiring small history Methods requiring large history Summary of solvers 1 / 52 Conjugate
More informationJordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS
Jordan Journal of Mathematics and Statistics JJMS) 53), 2012, pp.169-184 A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS ADEL H. AL-RABTAH Abstract. The Jacobi and Gauss-Seidel iterative
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 13
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 208 LECTURE 3 need for pivoting we saw that under proper circumstances, we can write A LU where 0 0 0 u u 2 u n l 2 0 0 0 u 22 u 2n L l 3 l 32, U 0 0 0 l n l
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method The minimization problem We are given a symmetric positive definite matrix R n n and a right hand side vector b R n We want to solve the linear system Find u R n such that
More information