Krylov Space Methods. Nonstationary sounds good. Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 1 / 17
|
|
- Nancy Horton
- 5 years ago
- Views:
Transcription
1 Krylov Space Methods Nonstationary sounds good Radu Trîmbiţaş Babeş-Bolyai University Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 1 / 17
2 Introduction These methods are used both to solve Ax = b and to nd eigenvalues of A They assume A is accessible via a black box subroutine that returns y = Az (and perhaps y = A T z if A is nonsymmetric) no direct access or manipulation of matrix entries is used. Reasons: This is the cheapest operation on sparse matrices A may not be represented as a matrix but as a subroutine for computing Ax There exists a variety of such methods depending on A and availability of A T Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 2 / 17
3 Extracting information via matrix-vector multiplication Given a vector b and a subroutine for computing Ax what can we deduce about A? Compute y 1 = b,..., y m = Ay m 1 = A m 1 y 1. Let K = [y 1,..., y m ]. A K = [Ay 1,..., Ay m 1, Ay m ] = [y 2,..., y m, A m y 1 ]. (1) Assume K nonsingular, let c = K 1 A m y 1 ; A K = K [e 2, e 3,..., e m, c] K C, or c c K 1 AK = C = c m Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 3 / 17
4 Extracting information via matrix-vector multiplication C upper Hessenberg; it is a companion matrix and its caracteristic polynomial is p(x) = x m + m i=1 c i x i 1 We can in principle nd the eigenvalues of A by nding the zeros of p(x) This simple form is not very useful Finding c requires m 1 matrix-vector multiplication by A and solving a linear system with K K is ill-conditioned equivalent to power method y i converge to a dominant ev of A; column of K tend to get more and more parallel We x this by replacing K with an orthogonal matrix Q such that the leading k columns of K and Q span the same space called a Krylov subspace. We will compute only as many leading columns of Q as needed (for the solution of Ax = b or Ax = λx) very few compare to matrix size Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 4 / 17
5 Extracting information via matrix-vector multiplication Let K = QR the QR decomposition of K K 1 AK = R 1 Q T A(QR) = C =) Q T AQ = RCR 1 H C and H are upper Hessenberg If A is symmetric, then H = Q T AQ is tridiagonal We compute columns of Q one at a time, rather than all of them, using the modi ed Gram-Schmidt algorithm (instead of Householder re ections). Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 5 / 17
6 Extracting information via matrix-vector multiplication Let Q n be the m n matrix with the rst n columns of AQ = QH, or AQ n = Q n+1 eh n : h 11 h 1n 4 A 5 4 q 1... q n 5 = 4 q 1... q n+1 5 h The nth column of AQ n = Q n+1 H n gives Aq n = h 1n q h nn q n + h n+1,n q n+1 q i are orthonormal, multiply both sides by q T j h n+1,n q T j Aq n = n+1 h in qj T q n = h j,n =) h n+1,n q n+1 = Aq n i=1 n h in q i i=1 Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 6 / 17
7 The Arnoldi algorithm Algorithm: Arnoldi Iteration for partial reduction to Hessenberg form b := arbitrary; q 1 := b/kbk; for n := 1, 2, 3,... do v := Aq n ; for j := 1 ton do h jn := q j v; v := v h jn q j ; h n+1,n := kvk; if h n+1,n = 0 then return q n+1 := v/h n+1,n ; Complexity n matrix-vector multiplications by A +O(n 2 m) other work Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 7 / 17
8 The Arnoldi algorithm If we stop the algorithm here, what we have learned about A? Let Q = [Q n, Q u ], where Q n = [q 1,..., q k ], Q u = [q n+1,..., q m ]; colums of Q u excepting q n+1 are unknown Then H = Q T AQ = [Q n, Q u ] T Q T A [Q n, Q u ] = n AQ n Qu T AQ n n m n n m n Hn H nu H un H u Qn T AQ u Qu T AQ u H n is upper Hessenberg, H nu has a single nonzero entry (h n+1,n ), H u, H u,n unknown Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 8 / 17
9 Symmetric Matrices and the Lanczos Iteration For symmetric A, H n reduces to tridiagonal T n, and q n+1 can be computed by a three-term recurrence: Aq n = β n 1 q n 1 + α n q n + β n q n+1 Algorithm: Lanczos Iteration β 0 := 0; q 0 := 0; b :=arbitrary; q 1 := b/kbk; for n := 1, 2, 3,... do v := Aq n ; {or Aq n β n 1 q n 1 for greater stability} α n := q T n v; v := v β n 1 q n 1 α n q n ; β n := kvk; if β n = 0 then return q n+1 := v/β n ; Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 9 / 17
10 Symmetric Matrices and the Lanczos Iteration If we stop the algorithm here, what we have learned about A? We have T = Q T AQ = [Q n, Q u ] T Q T A [Q n, Q u ] = n AQ n Qn T AQ u Qu T AQ n Qu T AQ u = n m n Tn T nu Tnu T T u n m n Tn T nu T un T u Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 10 / 17
11 Krylov subspace De nition The Krylov subspace K n (A, b) is b, Ab, A 2 b,..., A n 1 b. We shall write K n instead of K n (A, b) if A and b are implicit from the context. H n or T n is the projection of A onto the Krylov subspace K n. One can show that K n has dimension n i the Arnoldi or Lanczos algorithm can compute q n without quitting rst. We shall solve Ax = b and Ax = λx using only the information computed by n steps of Arnoldi or Lanczos algorithm We hope n m, so the algorithms are e cient Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 11 / 17
12 Krylov subspace For eigenvalues, if h n+1,n = 0, then H (or T ) is block upper triangular and eigenvalues of H n are eigenvalues of H, and therefore of A To obtain eigenvector of A we multiply eigenvectors of H n by Q n If h n+1,n is small ew and ev of H n are good approximations to the ew and ev of A Roundo errors cause a number of algorithms to behave entirely di erent from how they would in exact arithmetic. In particular Lanczos vectorscan loose their orthogonality and become almost linearly dependent. Researchers learned how to stabilize the algorithm or that the convergence occured despite instability. Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 12 / 17
13 Solving a system using Krylov subspace We express the solution as the best approximation in the Krylov subspace of the form x n = n z k q k = Q n z, k=1 where z = [z 1,..., z n ] T We have to de ne best. There are several natural but di erent de nitions, leading to di erent algorithms. Let x = A 1 b be the exact solution and r n = b Ax n be the residual 1 x n minimizes kx n xk 2 we do not have enough information Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 13 / 17
14 Solving a system using Krylov subspace We express the solution as the best approximation in the Krylov subspace of the form x n = n z k q k = Q n z, k=1 where z = [z 1,..., z n ] T We have to de ne best. There are several natural but di erent de nitions, leading to di erent algorithms. Let x = A 1 b be the exact solution and r n = b Ax n be the residual 1 x n minimizes kx n xk 2 we do not have enough information 2 x n minimizes kr n k 2 A symmetric MINRES (minimum residual), A nonsymmetric (generalized minimum residual) Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 13 / 17
15 Solving a system using Krylov subspace We express the solution as the best approximation in the Krylov subspace of the form x n = n z k q k = Q n z, k=1 where z = [z 1,..., z n ] T We have to de ne best. There are several natural but di erent de nitions, leading to di erent algorithms. Let x = A 1 b be the exact solution and r n = b Ax n be the residual 1 x n minimizes kx n xk 2 we do not have enough information 2 x n minimizes kr n k 2 A symmetric MINRES (minimum residual), A nonsymmetric (generalized minimum residual) 3 x n makes r n? K n, i.e. Q T n r n = 0 SYMMLQ and GMRES Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 13 / 17
16 Solving a system using Krylov subspace We express the solution as the best approximation in the Krylov subspace of the form x n = n z k q k = Q n z, k=1 where z = [z 1,..., z n ] T We have to de ne best. There are several natural but di erent de nitions, leading to di erent algorithms. Let x = A 1 b be the exact solution and r n = b Ax n be the residual 1 x n minimizes kx n xk 2 we do not have enough information 2 x n minimizes kr n k 2 A symmetric MINRES (minimum residual), A nonsymmetric (generalized minimum residual) 3 x n makes r n? K n, i.e. Q T n r n = 0 SYMMLQ and GMRES 4 When A is SPD, it de nes a norm krk A 1 = r T A 1 r 1/2. The best x n minimizes kr n k A 1. This norm is the same as kx n xk A conjugate gradient algorithm Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 13 / 17
17 Minimizing Residuals Generalized Minimal RESiduals iterative method for solving Ax = b Find x n 2 K n that minimizes kr n k = kb Ax n k This is a least squares problem: Find a vector c such that kak n c bk = minimum where K n is the m n Krylov matrix QR factorization could be used to solve for c, and x n = K n c In practice the columns of K n are ill-conditioned and an orthogonal basis is used instead, produced by Arnoldi iteration Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 14 / 17
18 Minimal Residual with Orthogonal Basis Instead of x n = K n c set x n = Q n y, where the orthogonal columns of Q n span K n, and solve kaq n y bk = minimum For the Arnoldi iteration we showed that AQ n = Q n+1 H n : Qn+1 H n y b = minimum Left multiplication by Qn+1 does not change the norm (since both vectors are in the column space Q n+1 ): H n y Qn+1b = minimum Finally, it is clear that Qn+1 b = kbk e 1: H n y kbk e 1 = minimum Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 15 / 17
19 The GMRES Algorithm High-level description of the algorithm: Algorithm: GMRES q 1 := b/kbk; for n := 1, 2, 3,... do < step n of Arnoldi iteration > Find y to minimize k H n y kbke 1 k = kr n k; x n := Q n y The residual kr n k does not need to be computed explicitly from x n Least squares problem has Hessenberg structure, solve with QR factorization of H n (computed by updating the factorization of H n 1 ) Memory and cost grow with n restart the algorithm by clearing accumulated data (might stagnate the method) Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 16 / 17
20 Convergence of GMRES Two obvious observations based on the minimization in K n : GMRES converges monotonically and it converges after at most m steps, kr n+1 k kr n k ^ kr m k = 0 The residual r n = p n (A)b, where p n 2 P n is a degree n polynomial with p(0) = 1, so GMRES also nds a minimizing polynomial: kp n (A)bk = minimum Based on this, diagonalizable A = V ΛV 1 converges as: kr n k kbk κ(v ) inf p n 2P n kp n k Λ(A) or in words: If A has well-conditioned eigenvectors, the convergence is based on how small polynomials p n can be on the spectrum Radu Trîmbiţaş ( Babeş-Bolyai University) Krylov Space Methods 17 / 17
4.8 Arnoldi Iteration, Krylov Subspaces and GMRES
48 Arnoldi Iteration, Krylov Subspaces and GMRES We start with the problem of using a similarity transformation to convert an n n matrix A to upper Hessenberg form H, ie, A = QHQ, (30) with an appropriate
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 9 Minimizing Residual CG
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More informationEigenvalue and Eigenvector Problems
Eigenvalue and Eigenvector Problems An attempt to introduce eigenproblems Radu Trîmbiţaş Babeş-Bolyai University April 8, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) Eigenvalue and Eigenvector Problems
More informationLast Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection
Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue
More informationChapter 7. Iterative methods for large sparse linear systems. 7.1 Sparse matrix algebra. Large sparse matrices
Chapter 7 Iterative methods for large sparse linear systems In this chapter we revisit the problem of solving linear systems of equations, but now in the context of large sparse systems. The price to pay
More informationCourse Notes: Week 1
Course Notes: Week 1 Math 270C: Applied Numerical Linear Algebra 1 Lecture 1: Introduction (3/28/11) We will focus on iterative methods for solving linear systems of equations (and some discussion of eigenvalues
More informationKrylov Subspace Methods that Are Based on the Minimization of the Residual
Chapter 5 Krylov Subspace Methods that Are Based on the Minimization of the Residual Remark 51 Goal he goal of these methods consists in determining x k x 0 +K k r 0,A such that the corresponding Euclidean
More information6.4 Krylov Subspaces and Conjugate Gradients
6.4 Krylov Subspaces and Conjugate Gradients Our original equation is Ax = b. The preconditioned equation is P Ax = P b. When we write P, we never intend that an inverse will be explicitly computed. P
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 19: More on Arnoldi Iteration; Lanczos Iteration Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 17 Outline 1
More informationApplied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.4: Krylov Subspace Methods 2 / 51 Krylov Subspace Methods In this chapter we give
More informationFEM and sparse linear system solving
FEM & sparse linear system solving, Lecture 9, Nov 19, 2017 1/36 Lecture 9, Nov 17, 2017: Krylov space methods http://people.inf.ethz.ch/arbenz/fem17 Peter Arbenz Computer Science Department, ETH Zürich
More information1 Extrapolation: A Hint of Things to Come
Notes for 2017-03-24 1 Extrapolation: A Hint of Things to Come Stationary iterations are simple. Methods like Jacobi or Gauss-Seidel are easy to program, and it s (relatively) easy to analyze their convergence.
More informationThe QR Factorization
The QR Factorization How to Make Matrices Nicer Radu Trîmbiţaş Babeş-Bolyai University March 11, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) The QR Factorization March 11, 2009 1 / 25 Projectors A projector
More informationKrylov Subspaces. Lab 1. The Arnoldi Iteration
Lab 1 Krylov Subspaces Lab Objective: Discuss simple Krylov Subspace Methods for finding eigenvalues and show some interesting applications. One of the biggest difficulties in computational linear algebra
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Arnoldi Notes for 2016-11-16 Krylov subspaces are good spaces for approximation schemes. But the power basis (i.e. the basis A j b for j = 0,..., k 1) is not good for numerical work. The vectors in the
More informationAlgorithms that use the Arnoldi Basis
AMSC 600 /CMSC 760 Advanced Linear Numerical Analysis Fall 2007 Arnoldi Methods Dianne P. O Leary c 2006, 2007 Algorithms that use the Arnoldi Basis Reference: Chapter 6 of Saad The Arnoldi Basis How to
More informationIterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)
Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential
More informationIterative Methods for Sparse Linear Systems
Iterative Methods for Sparse Linear Systems Luca Bergamaschi e-mail: berga@dmsa.unipd.it - http://www.dmsa.unipd.it/ berga Department of Mathematical Methods and Models for Scientific Applications University
More informationEigenvalue Problems. Eigenvalue problems occur in many areas of science and engineering, such as structural analysis
Eigenvalue Problems Eigenvalue problems occur in many areas of science and engineering, such as structural analysis Eigenvalues also important in analyzing numerical methods Theory and algorithms apply
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 4 Eigenvalue Problems Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationKrylov subspace projection methods
I.1.(a) Krylov subspace projection methods Orthogonal projection technique : framework Let A be an n n complex matrix and K be an m-dimensional subspace of C n. An orthogonal projection technique seeks
More informationComputation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated.
Math 504, Homework 5 Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated 1 Find the eigenvalues and the associated eigenspaces
More informationMATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.
MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v
More informationKrylov Subspaces. The order-n Krylov subspace of A generated by x is
Lab 1 Krylov Subspaces Lab Objective: matrices. Use Krylov subspaces to find eigenvalues of extremely large One of the biggest difficulties in computational linear algebra is the amount of memory needed
More informationON ORTHOGONAL REDUCTION TO HESSENBERG FORM WITH SMALL BANDWIDTH
ON ORTHOGONAL REDUCTION TO HESSENBERG FORM WITH SMALL BANDWIDTH V. FABER, J. LIESEN, AND P. TICHÝ Abstract. Numerous algorithms in numerical linear algebra are based on the reduction of a given matrix
More informationA DISSERTATION. Extensions of the Conjugate Residual Method. by Tomohiro Sogabe. Presented to
A DISSERTATION Extensions of the Conjugate Residual Method ( ) by Tomohiro Sogabe Presented to Department of Applied Physics, The University of Tokyo Contents 1 Introduction 1 2 Krylov subspace methods
More informationThe Lanczos and conjugate gradient algorithms
The Lanczos and conjugate gradient algorithms Gérard MEURANT October, 2008 1 The Lanczos algorithm 2 The Lanczos algorithm in finite precision 3 The nonsymmetric Lanczos algorithm 4 The Golub Kahan bidiagonalization
More informationSparse matrix methods in quantum chemistry Post-doctorale cursus quantumchemie Han-sur-Lesse, Belgium
Sparse matrix methods in quantum chemistry Post-doctorale cursus quantumchemie Han-sur-Lesse, Belgium Dr. ir. Gerrit C. Groenenboom 14 june - 18 june, 1993 Last revised: May 11, 2005 Contents 1 Introduction
More informationMath 504 (Fall 2011) 1. (*) Consider the matrices
Math 504 (Fall 2011) Instructor: Emre Mengi Study Guide for Weeks 11-14 This homework concerns the following topics. Basic definitions and facts about eigenvalues and eigenvectors (Trefethen&Bau, Lecture
More informationSummary of Iterative Methods for Non-symmetric Linear Equations That Are Related to the Conjugate Gradient (CG) Method
Summary of Iterative Methods for Non-symmetric Linear Equations That Are Related to the Conjugate Gradient (CG) Method Leslie Foster 11-5-2012 We will discuss the FOM (full orthogonalization method), CG,
More informationThe quadratic eigenvalue problem (QEP) is to find scalars λ and nonzero vectors u satisfying
I.2 Quadratic Eigenvalue Problems 1 Introduction The quadratic eigenvalue problem QEP is to find scalars λ and nonzero vectors u satisfying where Qλx = 0, 1.1 Qλ = λ 2 M + λd + K, M, D and K are given
More informationConjugate gradient method. Descent method. Conjugate search direction. Conjugate Gradient Algorithm (294)
Conjugate gradient method Descent method Hestenes, Stiefel 1952 For A N N SPD In exact arithmetic, solves in N steps In real arithmetic No guaranteed stopping Often converges in many fewer than N steps
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationIterative methods for Linear System
Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationSimple iteration procedure
Simple iteration procedure Solve Known approximate solution Preconditionning: Jacobi Gauss-Seidel Lower triangle residue use of pre-conditionner correction residue use of pre-conditionner Convergence Spectral
More informationAPPLIED NUMERICAL LINEAR ALGEBRA
APPLIED NUMERICAL LINEAR ALGEBRA James W. Demmel University of California Berkeley, California Society for Industrial and Applied Mathematics Philadelphia Contents Preface 1 Introduction 1 1.1 Basic Notation
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationCharles University Faculty of Mathematics and Physics DOCTORAL THESIS. Krylov subspace approximations in linear algebraic problems
Charles University Faculty of Mathematics and Physics DOCTORAL THESIS Iveta Hnětynková Krylov subspace approximations in linear algebraic problems Department of Numerical Mathematics Supervisor: Doc. RNDr.
More informationSome minimization problems
Week 13: Wednesday, Nov 14 Some minimization problems Last time, we sketched the following two-step strategy for approximating the solution to linear systems via Krylov subspaces: 1. Build a sequence of
More informationTopics. The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems
Topics The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems What about non-spd systems? Methods requiring small history Methods requiring large history Summary of solvers 1 / 52 Conjugate
More informationBindel, Spring 2016 Numerical Analysis (CS 4220) Notes for
Notes for 2016-03-11 Woah, We re Halfway There Last time, we showed that the QR iteration maps upper Hessenberg matrices to upper Hessenberg matrices, and this fact allows us to do one QR sweep in O(n
More informationLINEAR ALGEBRA 1, 2012-I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS
LINEAR ALGEBRA, -I PARTIAL EXAM SOLUTIONS TO PRACTICE PROBLEMS Problem (a) For each of the two matrices below, (i) determine whether it is diagonalizable, (ii) determine whether it is orthogonally diagonalizable,
More informationSOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA
1 SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA 2 OUTLINE Sparse matrix storage format Basic factorization
More informationA short course on: Preconditioned Krylov subspace methods. Yousef Saad University of Minnesota Dept. of Computer Science and Engineering
A short course on: Preconditioned Krylov subspace methods Yousef Saad University of Minnesota Dept. of Computer Science and Engineering Universite du Littoral, Jan 19-3, 25 Outline Part 1 Introd., discretization
More informationLecture # 11 The Power Method for Eigenvalues Part II. The power method find the largest (in magnitude) eigenvalue of. A R n n.
Lecture # 11 The Power Method for Eigenvalues Part II The power method find the largest (in magnitude) eigenvalue of It makes two assumptions. 1. A is diagonalizable. That is, A R n n. A = XΛX 1 for some
More informationLab 1: Iterative Methods for Solving Linear Systems
Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as
More informationNumerical Methods I: Eigenvalues and eigenvectors
1/25 Numerical Methods I: Eigenvalues and eigenvectors Georg Stadler Courant Institute, NYU stadler@cims.nyu.edu November 2, 2017 Overview 2/25 Conditioning Eigenvalues and eigenvectors How hard are they
More informationMatrices, Moments and Quadrature, cont d
Jim Lambers CME 335 Spring Quarter 2010-11 Lecture 4 Notes Matrices, Moments and Quadrature, cont d Estimation of the Regularization Parameter Consider the least squares problem of finding x such that
More informationLinear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions
Linear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2013 Main problem of linear algebra 2: Given
More information1 Conjugate gradients
Notes for 2016-11-18 1 Conjugate gradients We now turn to the method of conjugate gradients (CG), perhaps the best known of the Krylov subspace solvers. The CG iteration can be characterized as the iteration
More informationEigenvalues and eigenvectors
Chapter 6 Eigenvalues and eigenvectors An eigenvalue of a square matrix represents the linear operator as a scaling of the associated eigenvector, and the action of certain matrices on general vectors
More informationLarge-scale eigenvalue problems
ELE 538B: Mathematics of High-Dimensional Data Large-scale eigenvalue problems Yuxin Chen Princeton University, Fall 208 Outline Power method Lanczos algorithm Eigenvalue problems 4-2 Eigendecomposition
More informationLecture 17 Methods for System of Linear Equations: Part 2. Songting Luo. Department of Mathematics Iowa State University
Lecture 17 Methods for System of Linear Equations: Part 2 Songting Luo Department of Mathematics Iowa State University MATH 481 Numerical Methods for Differential Equations Songting Luo ( Department of
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method The minimization problem We are given a symmetric positive definite matrix R n n and a right hand side vector b R n We want to solve the linear system Find u R n such that
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 21: Sensitivity of Eigenvalues and Eigenvectors; Conjugate Gradient Method Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis
More informationLARGE SPARSE EIGENVALUE PROBLEMS. General Tools for Solving Large Eigen-Problems
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization General Tools for Solving Large Eigen-Problems
More informationMAT Linear Algebra Collection of sample exams
MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationMath 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm
Math 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm References: Trefethen & Bau textbook Eigenvalue problem: given a matrix A, find
More informationLARGE SPARSE EIGENVALUE PROBLEMS
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization 14-1 General Tools for Solving Large Eigen-Problems
More informationGMRES: Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
GMRES: Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University December 4, 2011 T.-M. Huang (NTNU) GMRES
More informationResearch Article Some Generalizations and Modifications of Iterative Methods for Solving Large Sparse Symmetric Indefinite Linear Systems
Abstract and Applied Analysis Article ID 237808 pages http://dxdoiorg/055/204/237808 Research Article Some Generalizations and Modifications of Iterative Methods for Solving Large Sparse Symmetric Indefinite
More informationWorksheet for Lecture 25 Section 6.4 Gram-Schmidt Process
Worksheet for Lecture Name: Section.4 Gram-Schmidt Process Goal For a subspace W = Span{v,..., v n }, we want to find an orthonormal basis of W. Example Let W = Span{x, x } with x = and x =. Give an orthogonal
More informationECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems. Part I: Review of basic theory of eigenvalue problems
ECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems Part I: Review of basic theory of eigenvalue problems 1. Let A C n n. (a) A scalar λ is an eigenvalue of an n n A
More informationApplied Linear Algebra
Applied Linear Algebra Gábor P. Nagy and Viktor Vígh University of Szeged Bolyai Institute Winter 2014 1 / 262 Table of contents I 1 Introduction, review Complex numbers Vectors and matrices Determinants
More informationEIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4
EIGENVALUE PROBLEMS EIGENVALUE PROBLEMS p. 1/4 EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue
More informationKrylov Subspace Methods for Large/Sparse Eigenvalue Problems
Krylov Subspace Methods for Large/Sparse Eigenvalue Problems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan April 17, 2012 T.-M. Huang (Taiwan Normal University) Krylov
More informationOn prescribing Ritz values and GMRES residual norms generated by Arnoldi processes
On prescribing Ritz values and GMRES residual norms generated by Arnoldi processes Jurjen Duintjer Tebbens Institute of Computer Science Academy of Sciences of the Czech Republic joint work with Gérard
More informationA QR-decomposition of block tridiagonal matrices generated by the block Lanczos process
1 A QR-decomposition of block tridiagonal matrices generated by the block Lanczos process Thomas Schmelzer Martin H Gutknecht Oxford University Computing Laboratory Seminar for Applied Mathematics, ETH
More informationIterative Methods for Linear Systems of Equations
Iterative Methods for Linear Systems of Equations Projection methods (3) ITMAN PhD-course DTU 20-10-08 till 24-10-08 Martin van Gijzen 1 Delft University of Technology Overview day 4 Bi-Lanczos method
More information1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det
What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix
More informationLecture 9: Krylov Subspace Methods. 2 Derivation of the Conjugate Gradient Algorithm
CS 622 Data-Sparse Matrix Computations September 19, 217 Lecture 9: Krylov Subspace Methods Lecturer: Anil Damle Scribes: David Eriksson, Marc Aurele Gilles, Ariah Klages-Mundt, Sophia Novitzky 1 Introduction
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Power iteration Notes for 2016-10-17 In most introductory linear algebra classes, one computes eigenvalues as roots of a characteristic polynomial. For most problems, this is a bad idea: the roots of
More informationIntroduction. Chapter One
Chapter One Introduction The aim of this book is to describe and explain the beautiful mathematical relationships between matrices, moments, orthogonal polynomials, quadrature rules and the Lanczos and
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationIDR(s) as a projection method
Delft University of Technology Faculty of Electrical Engineering, Mathematics and Computer Science Delft Institute of Applied Mathematics IDR(s) as a projection method A thesis submitted to the Delft Institute
More informationEECS 275 Matrix Computation
EECS 275 Matrix Computation Ming-Hsuan Yang Electrical Engineering and Computer Science University of California at Merced Merced, CA 95344 http://faculty.ucmerced.edu/mhyang Lecture 17 1 / 26 Overview
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationM.A. Botchev. September 5, 2014
Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev
More informationParallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1
Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations
More informationNotes on Eigenvalues, Singular Values and QR
Notes on Eigenvalues, Singular Values and QR Michael Overton, Numerical Computing, Spring 2017 March 30, 2017 1 Eigenvalues Everyone who has studied linear algebra knows the definition: given a square
More informationCheat Sheet for MATH461
Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A
More informationUpdating the QR decomposition of block tridiagonal and block Hessenberg matrices generated by block Krylov space methods
Updating the QR decomposition of block tridiagonal and block Hessenberg matrices generated by block Krylov space methods Martin H. Gutknecht Seminar for Applied Mathematics, ETH Zurich, ETH-Zentrum HG,
More informationOn the influence of eigenvalues on Bi-CG residual norms
On the influence of eigenvalues on Bi-CG residual norms Jurjen Duintjer Tebbens Institute of Computer Science Academy of Sciences of the Czech Republic duintjertebbens@cs.cas.cz Gérard Meurant 30, rue
More informationITERATIVE METHODS BASED ON KRYLOV SUBSPACES
ITERATIVE METHODS BASED ON KRYLOV SUBSPACES LONG CHEN We shall present iterative methods for solving linear algebraic equation Au = b based on Krylov subspaces We derive conjugate gradient (CG) method
More informationIndex. for generalized eigenvalue problem, butterfly form, 211
Index ad hoc shifts, 165 aggressive early deflation, 205 207 algebraic multiplicity, 35 algebraic Riccati equation, 100 Arnoldi process, 372 block, 418 Hamiltonian skew symmetric, 420 implicitly restarted,
More informationMA 265 FINAL EXAM Fall 2012
MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators
More informationSolution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method
Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts
More informationLinear Algebra, part 3. Going back to least squares. Mathematical Models, Analysis and Simulation = 0. a T 1 e. a T n e. Anna-Karin Tornberg
Linear Algebra, part 3 Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2010 Going back to least squares (Sections 1.7 and 2.3 from Strang). We know from before: The vector
More informationPreconditioned inverse iteration and shift-invert Arnoldi method
Preconditioned inverse iteration and shift-invert Arnoldi method Melina Freitag Department of Mathematical Sciences University of Bath CSC Seminar Max-Planck-Institute for Dynamics of Complex Technical
More informationModel reduction of large-scale dynamical systems
Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation Thanos Antoulas Rice University and Jacobs University email: aca@rice.edu URL: www.ece.rice.edu/
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationA stable variant of Simpler GMRES and GCR
A stable variant of Simpler GMRES and GCR Miroslav Rozložník joint work with Pavel Jiránek and Martin H. Gutknecht Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic miro@cs.cas.cz,
More informationContents. Preface... xi. Introduction...
Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures... 1 1.1. Different types of parallelism... 1 1.1.1. Overlap, concurrency and parallelism... 1 1.1.2. Temporal and spatial parallelism
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large
More informationInstitute for Advanced Computer Studies. Department of Computer Science. Iterative methods for solving Ax = b. GMRES/FOM versus QMR/BiCG
University of Maryland Institute for Advanced Computer Studies Department of Computer Science College Park TR{96{2 TR{3587 Iterative methods for solving Ax = b GMRES/FOM versus QMR/BiCG Jane K. Cullum
More informationNumerical Methods I Eigenvalue Problems
Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 2nd, 2014 A. Donev (Courant Institute) Lecture
More informationApproximating the matrix exponential of an advection-diffusion operator using the incomplete orthogonalization method
Approximating the matrix exponential of an advection-diffusion operator using the incomplete orthogonalization method Antti Koskela KTH Royal Institute of Technology, Lindstedtvägen 25, 10044 Stockholm,
More informationChasing the Bulge. Sebastian Gant 5/19/ The Reduction to Hessenberg Form 3
Chasing the Bulge Sebastian Gant 5/9/207 Contents Precursers and Motivation 2 The Reduction to Hessenberg Form 3 3 The Algorithm 5 4 Concluding Remarks 8 5 References 0 ntroduction n the early days of
More information