Computing Eigenvalues and/or Eigenvectors;Part 2, The Power method and QR-algorithm
|
|
- Kory Roberts
- 5 years ago
- Views:
Transcription
1 Computing Eigenvalues and/or Eigenvectors;Part 2, The Power method and QR-algorithm Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 19, 2010
2 Today The power method to find the dominant eigenvector The shifted power method to speed up convergence The inverse power method The Rayleigh quotient iteration The QR-algorithm
3 The Power Method Find the eigenvector corresponding to the dominant (largest in absolute value) eigenvalue. With a simple modification we can also find the corresponding eigenvalue
4 Assumptions Let A C n,n have eigenpairs (λ j, v j ), j = 1,..., n. Given z 0 C n we assume that (i) λ 1 > λ 2 λ 3 λ n, (ii) z T 0 v 1 0. (iii) A has linearly independent eigenvectors. The first assumption means that A has a dominant eigenvalue λ 1 of algebraic multiplicity one. The second assumption says that z 0 has a component in the direction v 1. The third assumption is not necessary. It is included to simplify the analysis.
5 Powers Given A C n,n, a vector z 0 C n, and assume that i),ii), iii) hold. Define a sequence {z k } of vectors in C n by z k := A k z 0 = Az k 1, k = 1, 2,.... z 0 = c 1 v 1 + c 2 v c n v n, with c 1 0. A k v j = λ k j v j, k = 0, 1, 2,..., j = 1,..., n. Then z k = c 1 A k v 1 + c 2 A k v c n A k v n, k = 0, 1, 2,.... z k = c 1 λ k 1v 1 + c 2 λ k 2v c n λ k nv n, k = 0, 1, 2,.... ( ) kv2 ( ) kvn λ = c 1 v 1 + c 2 λ 2 λ c n n λ 1. z k λ k 1 z k /λ k 1, c 1 v 1, k
6 The Power method Need to normalize the vectors z k. Do not know λ 1. Choose a norm on C n, set x 0 = z 0 / z 0 and generate for k = 1, 2,... unit vectors,{x k } as follows: (i) (ii) y k = Ax k 1 x k = y k / y k. (1)
7 Example1 A = [ ] 1 2, z = [1.0, 1.0] T, x 0 = [0.707, 0.707] T x 1 = [0.39, 0.92], x 2 = [0.4175, ], x 3 = [0.4159, ],... converges to an eigenvector of A
8 Example2 The way {x k } converges to an eigenvector can be more complicated. [ ] 1 2 A =, z = [1.0, 1.0] T, x 0 = [0.707, 0.707] T x 1 = [ 0.39, 0.92], x 2 = [0.4175, ], x 3 = [ , ],... changes sign in each iteration.
9 Convergence Lemma Suppose (i), (ii), (iii) hold. Then ( λ 1 ) kxk lim = c 1 k λ 1 c 1 v 1 v 1. In particular, if λ 1 > 0 and c 1 > 0 then the sequence {x k } will converge to the eigenvector u 1 := v 1 / v 1 of unit length. In Example 2, λ 1 < 0 and c 1 > 0 if u 1 = [0.4159, ] T. For k large: x k ( ) λ 1 k c1 u λ 1 c 1 1 = ( 1) k u 1.
10 Eigenvalue Suppose we know an approximate eigenvector of A C n,n. How should we estimate the corresponding eigenvalue? If (λ, u) is an exact eigenpair then Au λu = 0. If u is an approximate eigenvector we can minimize the function ρ : C R given by Theorem ρ(µ) := Au µu 2. ρ is minimized when µ = ν := u Au u u u. Proof on blackboard. is the Rayleigh quotient of
11 Power with Rayleigh function [l,x,it]=powerit(a,z,k,tol) af=norm(a, fro ); x=z/norm(z); for k=1:k y=a*x; l=x *y; if norm(y-l*x)/af < tol it=k; x=y/norm(y); return end x=y/norm(y); end it=k+1;
12 Example A 1 := [ ] 1 2, A := [ ] , and A = [ ] Start with a random vector and tol=10 6. Get convergence in 7 iterations for A 1, 174 iterations for A 2 and no convergence for A 3. A 3 has two complex eigenvalues so assumption i) is not satisfied Rate of convergence depends on r = λ 2 /λ 1. Faster convergence for smaller r. We have r 0.07 for A 1 and r = 0.95 for A 2.
13 The shifted power method A variant of the power method is the shifted power method. In this method we choose a number s and apply the power method to the matrix A si. The number s is called a shift since it shifts an eigenvalue λ of A to λ s of A si. Sometimes the convergence can be faster if the shift is chosen intelligently. For example, for A 2 with shift s = 1.8, we get convergence in 17 iterations instead of 174 without shift.
14 The inverse power method We apply the power method to the inverse matrix (A si) 1, where s is a shift. If A has eigenvalues λ 1,..., λ n in no particular order then (A si) 1 has eigenvalues µ 1 (s) = (λ 1 s) 1, µ 2 (s) = (λ 2 s) 1,..., µ n (s) = (λ n s) 1. Suppose λ 1 is a simple eigenvalue of A. Then lim s λ1 µ 1 (s) =, while lim s λ1 µ j (s) = (λ j λ 1 ) 1 < for j = 2,..., n. Hence, by choosing s sufficiently close to λ 1 the inverse power method will converge to that eigenvalue.
15 For the inverse power method (1) is replaced by. (i) (ii) (A si)y k = x k 1 x k = y k / y k. (2) Note that we solve the linear system rather than computing the inverse matrix. Normally the PLU-factorization of A si is pre-computed in order to speed up the iteration.
16 Rayleigh quotient iteration We can combine inverse power with Rayleigh quotient calculation. (i) (A s k 1 I)y k = x k 1, (ii) x k = y k / y k, (iii) s k = x kax k, (iv) r k = Ax k s k x k. We can avoid the calculation of Ax k in (iii) and (iv).
17 Example A 1 := [ ] Try to find the smallest eigenvalue λ = (5 33)/ Start with x = [1, 1] T and s = 0 k r 2 1.0e e e e e-020 s k λ 3.7e e e e e-016 Table: Quadratic convergence of Rayleigh quotient iteration
18 Problem with singularity? The linear system in i) becomes closer and closer to singular as s k converges to the eigenvalue. Thus the system becomes more and more ill-conditioned and we can expect large errors in the computed y k. This is indeed true, but we are lucky. Most of the error occurs in the direction of the eigenvector and this error disappears when we normalize y k in ii). Miraculously, the normalized eigenvector will be quite accurate.
19 Discussion Since the shift changes from iteration to iteration the computation of y will require O(n 3 ) flops for a full matrix. For such a matrix it might pay to reduce it to a upper Hessenberg form or tridiagonal form before starting the iteration. However, if we have a good approximation to an eigenpair then only a few iterations are necessary to obtain close to machine accuracy. If Rayleigh quotient iteration converges the convergence will be quadratic and sometimes even cubic.
20 The QR Algorithm An iterative method to compute all eigenvalues and eigenvectors of a matrix A C n,n. The matrix is reduced to triangular or quasitriangular form by a sequence of unitary similarity transformations computed from the QR factorization of A. Recall that for a square matrix the QR factorization and the QR decomposition are the same. If A = QR is a QR factorization then Q C n,n is unitary, Q Q = I and R C n,n is upper triangular.
21 Basic QR A 1 = A for k = 1, 2,... Q k R k = A k (QR factorization of A k ) A k+1 = R k Q k. end (3) The determination of the QR factorization of A k and the computation of R k Q k is called a QR step.
22 Example [ ] 2 1 A 1 = A = 1 2 A 1 = ( [ ] [ ] ) ( ) = Q1 R [ ] 5 4 A 2 = R 1 Q 1 = 1 5 [ ] [ ] A 4, [ A ] [ ]. = 1 5 [ ] 14 3 = 3 6 A 10 is almost diagonal and contains approximations to the eigenvalues λ 1 = 3 and λ 2 = 1 on the diagonal.
23 Example 2 we obtain A 14 = A 1 = A = e e e e e A 14 is close to quasi-triangular..
24 Example 2 A 14 = e e e e e The 1 1 blocks give us two real eigenvalues λ and λ The middle 2 2 block has complex eigenvalues resulting in λ i and λ i. From Gerschgorin s circle theorem it follows that the approximations to the real eigenvalues are quite accurate. We would also expect the complex eigenvalues to have small absolute errors.
25 Why QR works Since Q ka k = R k we obtain A k+1 = R k Q k = Q ka k Q k. (4) Thus A k+1 is similar to A k and hence to A. It combines both the power method and the Rayleigh quotient iteration. If A R n,n has real eigenvalues, then under fairly general conditions, the sequence {A k } converges to an upper triangular matrix, the Schur form. If A is real, but with some complex eigenvalues, then the convergence will be to the quasi-triangular Schur form
26 QR factorization of A k Theorem For k = 1, 2, 3,..., the QR factorization of A k is A k = Q k R k, where Q k := Q 1 Q k and R k := R k R 1, (5) and Q 1,..., Q k, R 1,..., R k are the matrices generated by the basic QR algorithm (3). Moreover, A k = Q k 1A Q k 1, k 1. (6) Proof on blackboard.
27 Relation to the Power Method Since R k is upper triangular its first column is a multiple of e 1 A k e 1 = Q k Rk e 1 = r (k) 11 Q k e 1 or q (k) 1 := Q k e 1 = 1 r (k) 11 A k e 1. Since q (k) 1 2 = 1 the first column of Q k is the result of applying the normalized power iteration to the starting vector x 0 = e 1. If this iteration converges we conclude that the first column of Q k must converge to a dominant eigenvector of A.
28 Initial reduction to Hessenberg form One QR step requires O(n 3 ) flops for a matrix A of order n. By an initial reduction of A to upper Hessenberg form H 1, the cost of a QR step can be reduced to O(n 2 ).
29 Invariance of the Hessenberg form; I Consider a QR step on H 1. We determine plane rotations P i,i+1, i = 1,..., n 1 so that P n 1,n P 1,2 H 1 = R 1 is upper triangular. x x x x x x x x 0 x x x 0 0 x x P 1,2 P 3,4 x x x x 0 x x x 0 x x x 0 0 x x x x x x 0 x x x 0 0 x x x P 2,3. x x x x 0 x x x 0 0 x x 0 0 x x
30 Invariance of the Hessenberg form; II H 1 = Q 1 R 1, where Q 1 = P 1,2 P n 1,n is a QR factorization of H 1. To finish the QR step we compute R 1 Q 1 = R 1 P 1,2 P n 1,n. This postmultiplication step is illustrated by the Wilkinson diagram ] [ P R 1 = x x x x ] [ 12 x x x x P x x x x ] 23 x x x x P 0 0 x x 34 0 x x x x x [ x x x x 0 x x x 0 0 x x x [ x x x x ] x x x x 0 x x x. 0 0 x x
31 Invariance of the Hessenberg form; III If A k is upper Hessenberg then A k+1 is upper Hessenberg. One QR step requires O(n 2 ) flops. If A is tridiagonal and symmetric then one QR step requires O(n) flops.
32 Deflation If a subdiagonal element a i+1,i of an upper Hessenberg matrix A is equal to zero, then the eigenvalues of A are the union of the eigenvalues of the two smaller matrices A(1 : i, 1 : i) and A(i + 1 : n, i + 1 : n). Thus if during the iteration the (i + 1, i) element of A k is sufficiently small then we can continue the iteration on the two smaller submatrices separately.
33 Effect on A of deflation suppose a (k) i+1,i ɛ. Â k := A k a (k) i+1,i e i+1e T i A k = Q k 1A Q k 1 Â k = Q k 1(A + E) Q k 1, E = Q k 1 (a (k) i+1,i e i+1e T i ) Q k 1. E F = a (k) i+1,i e i+1e T i F = a (k) i+1,i ɛ Setting a (k) i+1,i = 0 amounts to a perturbation in the original A of at most ɛ.
34 The Shifted QR algorithms Like in the inverse power method it is possible to speed up the convergence by introducing shifts. The explicitly shifted QR algorithm works as follows: A 1 = A for k = 1, 2,... end Choose a shift s k Q k R k = A k s k I A k+1 = R k Q k + s k I. Since R k = Q k(a k s k I) (QR factorization of A k si) (7) A k+1 = Q k(a k s k I)Q k + s k I = Q ka k Q k and A k+1 and A k are unitary similar.
35 Relation to the inverse power method A s k I = Q k R k (A s k I) = R kq k (A s k I) Q k = R k (A s k I) Q k e n = R ke n = r (k) nn e n. Q k e n is the result of one iteration of the inverse power method to A with shift s k.
36 Choice of shifts 1. The shift s k := e T n A k e n is called the Rayleigh quotient shift. 2. The eigenvalue of the lower right 2 2 corner of A k closest to the n, n element of A k is called the Wilkinson shift. This shift can be used to find complex eigenvalues of a real matrix. 3. The convergence is very fast and at least quadratic both for the Rayleigh quotient shift and the Wilkinson shift.
37 The Implicitly shifted QR algorithm 1. By doing two QR iterations at a time it is possible to find both real and complex eigenvalues without using complex arithmetic. The corresponding algorithm is called the implicitly shifted QR algorithm 2. After having computed the eigenvalues we can compute the eigenvectors in steps. First we find the eigenvectors of the triangular or quasi-triangular matrix. We then compute the eigenvectors of the upper Hessenberg matrix and finally we get the eigenvectors of A. 3. Practical experience indicates that only O(n) iterations are needed to find all eigenvalues of A. Thus both the explicit- and implicit shift QR algorithms are normally O(n 3 ) algorithms.
Computing Eigenvalues and/or Eigenvectors;Part 2, The Power method and QR-algorithm
Computing Eigenvalues and/or Eigenvectors;Part 2, The Power method and QR-algorithm Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 13, 2009 Today
More informationEIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4
EIGENVALUE PROBLEMS EIGENVALUE PROBLEMS p. 1/4 EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue
More informationQR-decomposition. The QR-decomposition of an n k matrix A, k n, is an n n unitary matrix Q and an n k upper triangular matrix R for which A = QR
QR-decomposition The QR-decomposition of an n k matrix A, k n, is an n n unitary matrix Q and an n k upper triangular matrix R for which In Matlab A = QR [Q,R]=qr(A); Note. The QR-decomposition is unique
More informationNumerical methods for eigenvalue problems
Numerical methods for eigenvalue problems D. Löchel Supervisors: M. Hochbruck und M. Tokar Mathematisches Institut Heinrich-Heine-Universität Düsseldorf GRK 1203 seminar february 2008 Outline Introduction
More informationOrthogonal iteration to QR
Notes for 2016-03-09 Orthogonal iteration to QR The QR iteration is the workhorse for solving the nonsymmetric eigenvalue problem. Unfortunately, while the iteration itself is simple to write, the derivation
More informationEigenvalues and eigenvectors
Chapter 6 Eigenvalues and eigenvectors An eigenvalue of a square matrix represents the linear operator as a scaling of the associated eigenvector, and the action of certain matrices on general vectors
More informationComputing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices
Computing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 8, 2009 Today
More informationComputation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated.
Math 504, Homework 5 Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated 1 Find the eigenvalues and the associated eigenspaces
More informationECS130 Scientific Computing Handout E February 13, 2017
ECS130 Scientific Computing Handout E February 13, 2017 1. The Power Method (a) Pseudocode: Power Iteration Given an initial vector u 0, t i+1 = Au i u i+1 = t i+1 / t i+1 2 (approximate eigenvector) θ
More informationSolving large scale eigenvalue problems
arge scale eigenvalue problems, Lecture 5, March 23, 2016 1/30 Lecture 5, March 23, 2016: The QR algorithm II http://people.inf.ethz.ch/arbenz/ewp/ Peter Arbenz Computer Science Department, ETH Zürich
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Reduction to Hessenberg and Tridiagonal Forms; Rayleigh Quotient Iteration Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationMath 504 (Fall 2011) 1. (*) Consider the matrices
Math 504 (Fall 2011) Instructor: Emre Mengi Study Guide for Weeks 11-14 This homework concerns the following topics. Basic definitions and facts about eigenvalues and eigenvectors (Trefethen&Bau, Lecture
More informationSolving large scale eigenvalue problems
arge scale eigenvalue problems, Lecture 4, March 14, 2018 1/41 Lecture 4, March 14, 2018: The QR algorithm http://people.inf.ethz.ch/arbenz/ewp/ Peter Arbenz Computer Science Department, ETH Zürich E-mail:
More informationNumerical Methods I Eigenvalue Problems
Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 2nd, 2014 A. Donev (Courant Institute) Lecture
More informationEIGENVALUE PROBLEMS (EVP)
EIGENVALUE PROBLEMS (EVP) (Golub & Van Loan: Chaps 7-8; Watkins: Chaps 5-7) X.-W Chang and C. C. Paige PART I. EVP THEORY EIGENVALUES AND EIGENVECTORS Let A C n n. Suppose Ax = λx with x 0, then x is a
More informationComputing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices
Computing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 9, 2008 Today
More information5.3 The Power Method Approximation of the Eigenvalue of Largest Module
192 5 Approximation of Eigenvalues and Eigenvectors 5.3 The Power Method The power method is very good at approximating the extremal eigenvalues of the matrix, that is, the eigenvalues having largest and
More informationEigenvalue and Eigenvector Problems
Eigenvalue and Eigenvector Problems An attempt to introduce eigenproblems Radu Trîmbiţaş Babeş-Bolyai University April 8, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) Eigenvalue and Eigenvector Problems
More informationEECS 275 Matrix Computation
EECS 275 Matrix Computation Ming-Hsuan Yang Electrical Engineering and Computer Science University of California at Merced Merced, CA 95344 http://faculty.ucmerced.edu/mhyang Lecture 17 1 / 26 Overview
More informationMatrix Algorithms. Volume II: Eigensystems. G. W. Stewart H1HJ1L. University of Maryland College Park, Maryland
Matrix Algorithms Volume II: Eigensystems G. W. Stewart University of Maryland College Park, Maryland H1HJ1L Society for Industrial and Applied Mathematics Philadelphia CONTENTS Algorithms Preface xv xvii
More informationLecture 2 INF-MAT : , LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky
Lecture 2 INF-MAT 4350 2009: 7.1-7.6, LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky Tom Lyche and Michael Floater Centre of Mathematics for Applications, Department of Informatics,
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Solving Eigenvalue Problems All
More informationEigenvalues and Eigenvectors
Chapter 1 Eigenvalues and Eigenvectors Among problems in numerical linear algebra, the determination of the eigenvalues and eigenvectors of matrices is second in importance only to the solution of linear
More informationLinear algebra & Numerical Analysis
Linear algebra & Numerical Analysis Eigenvalues and Eigenvectors Marta Jarošová http://homel.vsb.cz/~dom033/ Outline Methods computing all eigenvalues Characteristic polynomial Jacobi method for symmetric
More informationIndex. for generalized eigenvalue problem, butterfly form, 211
Index ad hoc shifts, 165 aggressive early deflation, 205 207 algebraic multiplicity, 35 algebraic Riccati equation, 100 Arnoldi process, 372 block, 418 Hamiltonian skew symmetric, 420 implicitly restarted,
More informationNumerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
More informationEIGENVALUE PROBLEMS. Background on eigenvalues/ eigenvectors / decompositions. Perturbation analysis, condition numbers..
EIGENVALUE PROBLEMS Background on eigenvalues/ eigenvectors / decompositions Perturbation analysis, condition numbers.. Power method The QR algorithm Practical QR algorithms: use of Hessenberg form and
More informationMatrices, Moments and Quadrature, cont d
Jim Lambers CME 335 Spring Quarter 2010-11 Lecture 4 Notes Matrices, Moments and Quadrature, cont d Estimation of the Regularization Parameter Consider the least squares problem of finding x such that
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationLecture # 11 The Power Method for Eigenvalues Part II. The power method find the largest (in magnitude) eigenvalue of. A R n n.
Lecture # 11 The Power Method for Eigenvalues Part II The power method find the largest (in magnitude) eigenvalue of It makes two assumptions. 1. A is diagonalizable. That is, A R n n. A = XΛX 1 for some
More informationOrthogonal iteration to QR
Week 10: Wednesday and Friday, Oct 24 and 26 Orthogonal iteration to QR On Monday, we went through a somewhat roundabout algbraic path from orthogonal subspace iteration to the QR iteration. Let me start
More informationNumerical Methods I: Eigenvalues and eigenvectors
1/25 Numerical Methods I: Eigenvalues and eigenvectors Georg Stadler Courant Institute, NYU stadler@cims.nyu.edu November 2, 2017 Overview 2/25 Conditioning Eigenvalues and eigenvectors How hard are they
More informationKrylov subspace projection methods
I.1.(a) Krylov subspace projection methods Orthogonal projection technique : framework Let A be an n n complex matrix and K be an m-dimensional subspace of C n. An orthogonal projection technique seeks
More informationLecture Notes for Inf-Mat 3350/4350, Tom Lyche
Lecture Notes for Inf-Mat 3350/4350, 2007 Tom Lyche August 5, 2007 2 Contents Preface vii I A Review of Linear Algebra 1 1 Introduction 3 1.1 Notation............................... 3 2 Vectors 5 2.1 Vector
More informationEigenvalue Problems. Eigenvalue problems occur in many areas of science and engineering, such as structural analysis
Eigenvalue Problems Eigenvalue problems occur in many areas of science and engineering, such as structural analysis Eigenvalues also important in analyzing numerical methods Theory and algorithms apply
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 4 Eigenvalue Problems Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationLast Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection
Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue
More informationMath 411 Preliminaries
Math 411 Preliminaries Provide a list of preliminary vocabulary and concepts Preliminary Basic Netwon s method, Taylor series expansion (for single and multiple variables), Eigenvalue, Eigenvector, Vector
More information1 Number Systems and Errors 1
Contents 1 Number Systems and Errors 1 1.1 Introduction................................ 1 1.2 Number Representation and Base of Numbers............. 1 1.2.1 Normalized Floating-point Representation...........
More informationMath 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm
Math 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm References: Trefethen & Bau textbook Eigenvalue problem: given a matrix A, find
More informationOrthonormal Transformations and Least Squares
Orthonormal Transformations and Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 30, 2009 Applications of Qx with Q T Q = I 1. solving
More informationThe German word eigen is cognate with the Old English word āgen, which became owen in Middle English and own in modern English.
Chapter 4 EIGENVALUE PROBLEM The German word eigen is cognate with the Old English word āgen, which became owen in Middle English and own in modern English. 4.1 Mathematics 4.2 Reduction to Upper Hessenberg
More informationThe QR Algorithm. Marco Latini. February 26, 2004
The QR Algorithm Marco Latini February 26, 2004 Contents 1 Introduction 2 2 The Power and Inverse Power method 2 2.1 The Power Method............................... 2 2.2 The Inverse power method...........................
More information11.3 Eigenvalues and Eigenvectors of a Tridiagonal Matrix
11.3 Eigenvalues and Eigenvectors of a ridiagonal Matrix Evaluation of the Characteristic Polynomial Once our original, real, symmetric matrix has been reduced to tridiagonal form, one possible way to
More informationLecture 2: Numerical linear algebra
Lecture 2: Numerical linear algebra QR factorization Eigenvalue decomposition Singular value decomposition Conditioning of a problem Floating point arithmetic and stability of an algorithm Linear algebra
More informationLecture 4 Basic Iterative Methods I
March 26, 2018 Lecture 4 Basic Iterative Methods I A The Power Method Let A be an n n with eigenvalues λ 1,...,λ n counted according to multiplicity. We assume the eigenvalues to be ordered in absolute
More informationIntroduction to Applied Linear Algebra with MATLAB
Sigam Series in Applied Mathematics Volume 7 Rizwan Butt Introduction to Applied Linear Algebra with MATLAB Heldermann Verlag Contents Number Systems and Errors 1 1.1 Introduction 1 1.2 Number Representation
More informationOrthogonal Transformations
Orthogonal Transformations Tom Lyche University of Oslo Norway Orthogonal Transformations p. 1/3 Applications of Qx with Q T Q = I 1. solving least squares problems (today) 2. solving linear equations
More informationLecture 4 Eigenvalue problems
Lecture 4 Eigenvalue problems Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn
More informationIn this section again we shall assume that the matrix A is m m, real and symmetric.
84 3. The QR algorithm without shifts See Chapter 28 of the textbook In this section again we shall assume that the matrix A is m m, real and symmetric. 3.1. Simultaneous Iterations algorithm Suppose we
More informationThe QR Decomposition
The QR Decomposition We have seen one major decomposition of a matrix which is A = LU (and its variants) or more generally PA = LU for a permutation matrix P. This was valid for a square matrix and aided
More informationComputational Methods CMSC/AMSC/MAPL 460. EigenValue decomposition Singular Value Decomposition. Ramani Duraiswami, Dept. of Computer Science
Computational Methods CMSC/AMSC/MAPL 460 EigenValue decomposition Singular Value Decomposition Ramani Duraiswami, Dept. of Computer Science Hermitian Matrices A square matrix for which A = A H is said
More informationCHAPTER 5. Basic Iterative Methods
Basic Iterative Methods CHAPTER 5 Solve Ax = f where A is large and sparse (and nonsingular. Let A be split as A = M N in which M is nonsingular, and solving systems of the form Mz = r is much easier than
More informationImplementing the QR algorithm for efficiently computing matrix eigenvalues and eigenvectors
Implementing the QR algorithm for efficiently computing matrix eigenvalues and eigenvectors Final Degree Dissertation Degree in Mathematics Gorka Eraña Robles Supervisor: Ion Zaballa Tejada Leioa, June
More informationOrthonormal Transformations
Orthonormal Transformations Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 25, 2010 Applications of transformation Q : R m R m, with Q T Q = I 1.
More informationLecture 2 INF-MAT : A boundary value problem and an eigenvalue problem; Block Multiplication; Tridiagonal Systems
Lecture 2 INF-MAT 4350 2008: A boundary value problem and an eigenvalue problem; Block Multiplication; Tridiagonal Systems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University
More informationSection 4.5 Eigenvalues of Symmetric Tridiagonal Matrices
Section 4.5 Eigenvalues of Symmetric Tridiagonal Matrices Key Terms Symmetric matrix Tridiagonal matrix Orthogonal matrix QR-factorization Rotation matrices (plane rotations) Eigenvalues We will now complete
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J Olver 8 Numerical Computation of Eigenvalues In this part, we discuss some practical methods for computing eigenvalues and eigenvectors of matrices Needless to
More informationDouble-shift QR steps
Double-shift QR steps Week 11: Monday, Oct 29 Last time, we discussed the Wilkinson strategy of choosing as a shift one of the roots of the trailing 2-by-2 submatrix of A (k) (the one closest to the final
More informationEigenvalue Problems and Singular Value Decomposition
Eigenvalue Problems and Singular Value Decomposition Sanzheng Qiao Department of Computing and Software McMaster University August, 2012 Outline 1 Eigenvalue Problems 2 Singular Value Decomposition 3 Software
More informationECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems. Part I: Review of basic theory of eigenvalue problems
ECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems Part I: Review of basic theory of eigenvalue problems 1. Let A C n n. (a) A scalar λ is an eigenvalue of an n n A
More informationLecture 1 INF-MAT3350/ : Some Tridiagonal Matrix Problems
Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems Tom Lyche University of Oslo Norway Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.1/33 Plan for the day 1. Notation
More informationDELFT UNIVERSITY OF TECHNOLOGY
DELFT UNIVERSITY OF TECHNOLOGY REPORT -09 Computational and Sensitivity Aspects of Eigenvalue-Based Methods for the Large-Scale Trust-Region Subproblem Marielba Rojas, Bjørn H. Fotland, and Trond Steihaug
More informationRadial Basis Functions I
Radial Basis Functions I Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 14, 2008 Today Reformulation of natural cubic spline interpolation Scattered
More informationSolution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method
Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts
More informationChasing the Bulge. Sebastian Gant 5/19/ The Reduction to Hessenberg Form 3
Chasing the Bulge Sebastian Gant 5/9/207 Contents Precursers and Motivation 2 The Reduction to Hessenberg Form 3 3 The Algorithm 5 4 Concluding Remarks 8 5 References 0 ntroduction n the early days of
More informationACM106a - Homework 4 Solutions
ACM106a - Homework 4 Solutions prepared by Svitlana Vyetrenko November 17, 2006 1. Problem 1: (a) Let A be a normal triangular matrix. Without the loss of generality assume that A is upper triangular,
More informationLinear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions
Linear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2013 Main problem of linear algebra 2: Given
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationAPPLIED NUMERICAL LINEAR ALGEBRA
APPLIED NUMERICAL LINEAR ALGEBRA James W. Demmel University of California Berkeley, California Society for Industrial and Applied Mathematics Philadelphia Contents Preface 1 Introduction 1 1.1 Basic Notation
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 19: More on Arnoldi Iteration; Lanczos Iteration Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 17 Outline 1
More informationData Analysis and Manifold Learning Lecture 2: Properties of Symmetric Matrices and Examples
Data Analysis and Manifold Learning Lecture 2: Properties of Symmetric Matrices and Examples Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline
More informationThe Singular Value Decomposition and Least Squares Problems
The Singular Value Decomposition and Least Squares Problems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 27, 2009 Applications of SVD solving
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Algorithms Notes for 2016-10-31 There are several flavors of symmetric eigenvalue solvers for which there is no equivalent (stable) nonsymmetric solver. We discuss four algorithmic ideas: the workhorse
More informationNumerical Solution of Linear Eigenvalue Problems
Numerical Solution of Linear Eigenvalue Problems Jessica Bosch and Chen Greif Abstract We review numerical methods for computing eigenvalues of matrices We start by considering the computation of the dominant
More information13-2 Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2
The QR algorithm The most common method for solving small (dense) eigenvalue problems. The basic algorithm: QR without shifts 1. Until Convergence Do: 2. Compute the QR factorization A = QR 3. Set A :=
More informationLeast Squares. Tom Lyche. October 26, Centre of Mathematics for Applications, Department of Informatics, University of Oslo
Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 26, 2010 Linear system Linear system Ax = b, A C m,n, b C m, x C n. under-determined
More informationArnoldi Methods in SLEPc
Scalable Library for Eigenvalue Problem Computations SLEPc Technical Report STR-4 Available at http://slepc.upv.es Arnoldi Methods in SLEPc V. Hernández J. E. Román A. Tomás V. Vidal Last update: October,
More informationMAA507, Power method, QR-method and sparse matrix representation.
,, and representation. February 11, 2014 Lecture 7: Overview, Today we will look at:.. If time: A look at representation and fill in. Why do we need numerical s? I think everyone have seen how time consuming
More informationLecture 12 Eigenvalue Problem. Review of Eigenvalues Some properties Power method Shift method Inverse power method Deflation QR Method
Lecture Eigenvalue Problem Review of Eigenvalues Some properties Power method Shift method Inverse power method Deflation QR Method Eigenvalue Eigenvalue ( A I) If det( A I) (trivial solution) To obtain
More informationLinear Algebra Methods for Data Mining
Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 The Singular Value Decomposition (SVD) continued Linear Algebra Methods for Data Mining, Spring 2007, University
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationLecture 1 INF-MAT : Chapter 2. Examples of Linear Systems
Lecture 1 INF-MAT 4350 2010: Chapter 2. Examples of Linear Systems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo August 26, 2010 Notation The set of natural
More informationThe Eigenvalue Problem: Perturbation Theory
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 13 Notes These notes correspond to Sections 7.2 and 8.1 in the text. The Eigenvalue Problem: Perturbation Theory The Unsymmetric Eigenvalue Problem Just
More informationApplied Linear Algebra
Applied Linear Algebra Peter J. Olver School of Mathematics University of Minnesota Minneapolis, MN 55455 olver@math.umn.edu http://www.math.umn.edu/ olver Chehrzad Shakiban Department of Mathematics University
More informationDirect methods for symmetric eigenvalue problems
Direct methods for symmetric eigenvalue problems, PhD McMaster University School of Computational Engineering and Science February 4, 2008 1 Theoretical background Posing the question Perturbation theory
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Eigenvalue Problems; Similarity Transformations Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 18 Eigenvalue
More informationMATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.
MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More information5 Selected Topics in Numerical Linear Algebra
5 Selected Topics in Numerical Linear Algebra In this chapter we will be concerned mostly with orthogonal factorizations of rectangular m n matrices A The section numbers in the notes do not align with
More informationLeast squares and Eigenvalues
Lab 1 Least squares and Eigenvalues Lab Objective: Use least squares to fit curves to data and use QR decomposition to find eigenvalues. Least Squares A linear system Ax = b is overdetermined if it has
More informationEigenvalue problems. Eigenvalue problems
Determination of eigenvalues and eigenvectors Ax x, where A is an N N matrix, eigenvector x 0, and eigenvalues are in general complex numbers In physics: - Energy eigenvalues in a quantum mechanical system
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationEigensystems for Large Matrices
Eigensystems for Large Matrices L. Harris I. Papanikolaou J. Shi D. Strott Y. Tan C. Zhong Dr. Changhui Tan AMSC460 - Spring 205 May 2, 205 Abstract In this paper we will introduce the ideas behind several
More informationEigenpairs and Similarity Transformations
CHAPTER 5 Eigenpairs and Similarity Transformations Exercise 56: Characteristic polynomial of transpose we have that A T ( )=det(a T I)=det((A I) T )=det(a I) = A ( ) A ( ) = det(a I) =det(a T I) =det(a
More informationKrylov Subspace Methods to Calculate PageRank
Krylov Subspace Methods to Calculate PageRank B. Vadala-Roth REU Final Presentation August 1st, 2013 How does Google Rank Web Pages? The Web The Graph (A) Ranks of Web pages v = v 1... Dominant Eigenvector
More information632 CHAP. 11 EIGENVALUES AND EIGENVECTORS. QR Method
632 CHAP 11 EIGENVALUES AND EIGENVECTORS QR Method Suppose that A is a real symmetric matrix In the preceding section we saw how Householder s method is used to construct a similar tridiagonal matrix The
More informationNext topics: Solving systems of linear equations
Next topics: Solving systems of linear equations 1 Gaussian elimination (today) 2 Gaussian elimination with partial pivoting (Week 9) 3 The method of LU-decomposition (Week 10) 4 Iterative techniques:
More informationTHE QR METHOD A = Q 1 R 1
THE QR METHOD Given a square matrix A, form its QR factorization, as Then define A = Q 1 R 1 A 2 = R 1 Q 1 Continue this process: for k 1(withA 1 = A), A k = Q k R k A k+1 = R k Q k Then the sequence {A
More information