Today: eigenvalue sensitivity, eigenvalue algorithms Reminder: midterm starts today


 Stephanie Gardner
 1 years ago
 Views:
Transcription
1 AM 205: lecture 22 Today: eigenvalue sensitivity, eigenvalue algorithms Reminder: midterm starts today Posted online at 5 PM on Thursday 13th Deadline at 5 PM on Friday 14th Covers material up to and including lecture 16 Open book. No collaboration allowed. Send questions as private messages on Piazza
2 Gershgorin s Theorem (from last time) Theorem: All eigenvalues of A C n n are contained within the union of the n Gershgorin disks of A
3 Sensitivity of Eigenvalue Problems We shall now consider the sensitivity of the eigenvalues to perturbations in the matrix A Suppose A is nondefective, and hence A = VDV 1 Let δa denote a perturbation of A, and let E V 1 δav, then V 1 (A + δa)v = V 1 AV + V 1 δav = D + E
4 Sensitivity of Eigenvalue Problems For a nonsingular matrix X, the map A X 1 AX is called a similarity transformation of A Theorem: A similarity transformation preserves eigenvalues Proof: We can equate the characteristic polynomials of A and X 1 AX (denoted p A (z) and p X 1 AX (z), respectively) as follows: p X 1 AX (z) = det(zi X 1 AX ) = det(x 1 (zi A)X ) = det(x 1 ) det(zi A) det(x ) = det(zi A) = p A (z), where we have used the identities det(ab) = det(a) det(b), and det(x 1 ) = 1/ det(x )
5 Sensitivity of Eigenvalue Problems The identity V 1 (A + δa)v = D + E is a similarity transformation Therefore A + δa and D + E have the same eigenvalues Let λ k, k = 1, 2,..., n denote the eigenvalues of A, and λ denote an eigenvalue of A + δa Then for some w C n, ( λ, w) is an eigenpair of (D + E), i.e. (D + E)w = λw
6 Sensitivity of Eigenvalue Problems This can be rewritten as w = ( λi D) 1 Ew This is a promising start because: we want to bound λ λ k for some k ( λi D) 1 is a diagonal matrix with entries 1/( λ λ k ) on the diagonal
7 Sensitivity of Eigenvalue Problems Taking norms yields w 2 ( λi D) 1 2 E 2 w 2, or ( λi D) E 2 Note that the norm of a diagonal matrix is given by its largest entry (in abs. val.) 1 max v 0 Dv v (D 11 v 1, D 22 v 2,..., D nn v n ) = max v 0 v { } max D v ii max i=1,2,...,n v 0 v = max D ii i=1,2,...,n 1 This holds for any induced matrix norm, not just the 2norm
8 Sensitivity of Eigenvalue Problems Hence ( λi D) 1 2 = 1/ λ λ k, where λ k of A closest to λ is the eigenvalue Therefore it follows from ( λi D) E 2 that λ λ k = ( λi D) E 2 = V 1 δav 2 V 1 2 δa 2 V 2 = cond(v ) δa 2 This result is known as the Bauer Fike Theorem
9 Sensitivity of Eigenvalue Problems Hence suppose we compute the eigenvalues, λ i, of the perturbed matrix A + δa Then Bauer Fike tells us that each λ i must reside in a disk of radius cond(v ) δa 2 centered on some eigenvalue of A If V is poorly conditioned, then even for small perturbations δa, the disks can be large: sensitivity to perturbations If A is normal then cond(v ) = 1, in which case the Bauer Fike disk radius is just δa 2
10 Sensitivity of Eigenvalue Problems Note that a limitation of Bauer Fike is that it does not tell us which disk λ i will reside in Therefore, this doesn t rule out the possibility of, say, all λ i clustering in just one Bauer Fike disk In the case that A and A + δa are hermitian, we have a stronger result
11 Sensitivity of Eigenvalue Problems Weyl s Theorem: Let λ 1 λ 2 λ n and λ 1 λ 2 λ n be the eigenvalues of hermitian matrices A and A + δa, respectively. Then max λ i λ i δa 2. i=1,...,n Hence in the hermitian case, each perturbed eigenvalue must be in the disk 2 of its corresponding unperturbed eigenvalue! 2 In fact, eigenvalues of a hermitian matrix are real, so disk here is actually an interval in R
12 Sensitivity of Eigenvalue Problems The Bauer Fike Theorem relates to perturbations of the whole spectrum We can also consider perturbations of individual eigenvalues Suppose, for simplicity, that A C n n is symmetric, and consider the perturbed eigenvalue problem (A + E)(v + v) = (λ + λ)(v + v) Expanding this equation, dropping second order terms, and using Av = λv gives A v + Ev λv + λ v
13 Sensitivity of Eigenvalue Problems Premultiply A v + Ev λv + λ v by v to obtain v A v + v Ev λv v + λv v Noting that v A v = (v A v) = v Av = λ v v = λv v leads to v Ev λv v, or λ = v Ev v v
14 Sensitivity of Eigenvalue Problems Finally, we obtain λ v Ev v 2 2 v 2 Ev 2 v 2 2 = E 2, so that λ E 2 We observe that perturbation bound does not depend on cond(v ) when we consider only an individual eigenvalue this individual eigenvalue perturbation bound is asymptotic; it is rigorous only in the limit that the perturbations 0
15 Algorithms for Eigenvalue Problems
16 Power Method
17 Power Method The power method is perhaps the simplest eigenvalue algorithm It finds the eigenvalue of A C n n with largest modulus 1: choose x 0 C n arbitrarily 2: for k = 1, 2,... do 3: x k = Ax k 1 4: end for Question: How does this algorithm work?
18 Power Method Assuming A is nondefective, then the eigenvectors v 1, v 2,..., v n provide a basis for C n Therefore there exist coefficients α i such that x 0 = n j=1 α jv j Then, we have x k = Ax k 1 = A 2 x k 2 = = A k x 0 = n n A k α j v j = α j A k v j = j=1 n α j λ k j v j j=1 j=1 j=1 n 1 [ ] k = λ k λj n α n v n + α j v j λ n
19 Power Method Then if λ n > λ j, 1 j < n, we see that x k λ k nα n v n as k This algorithm converges linearly: the error terms are scaled by a factor at most λ n 1 / λ n at each iteration Also, we see that the method converges faster if λ n is wellseparated from the rest of the spectrum
20 Power Method However, in practice the exponential factor λ k n could cause overflow or underflow after relatively few iterations Therefore the standard form of the power method is actually the normalized power method 1: choose x 0 C n arbitrarily 2: for k = 1, 2,... do 3: y k = Ax k 1 4: x k = y k / y k 5: end for
21 Power Method Convergence analysis of the normalized power method is essentially the same as the unnormalized case Only difference is we now get an extra scaling factor, c k R, due to the normalization at each step n 1 [ ] k x k = c k λ k λj n α n v n + α j v j λ n j=1
22 Power Method This algorithm directly produces the eigenvector v n One way to recover λ n is to note that y k = Ax k 1 λ n x k 1 Hence we can compare an entry of y k and x k 1 to approximate λ n We also note two potential issues: 1. We require x 0 to have a nonzero component of v n 2. There may be more than one eigenvalue with maximum modulus
23 Power Method Issue 1: In practice, very unlikely that x 0 will be orthogonal to v n Even if x 0 v n = 0, rounding error will introduce a component of v n during the power iterations Issue 2: We cannot ignore the possibility that there is more than one max. eigenvalue In this case x k would converge to a member of the corresponding eigenspace
24 Power Method An important idea in eigenvalue computations is to consider the shifted matrix A σi, for σ R We see that (A σi)v i = (λ i σ)v i and hence the spectrum of A σi is shifted by σ, and the eigenvectors are the same For example, if all the eigenvalues are real, a shift can be used with the power method to converge to λ 1 instead of λ n
25 Power Method Python example: Consider power method and shifted power method for [ ] 4 1 A =, 1 2 which has eigenvalues λ 1 = , λ 2 =
26 Inverse Iteration
27 Inverse Iteration The eigenvalues of A 1 are the reciprocals of the eigenvalues of A, since Av = λv A 1 v = 1 λ v Question: What happens if we apply the power method to A 1?
28 Inverse Iteration Answer: We converge to the largest (in modulus) eigenvalue of A 1, which is 1/λ 1 (recall that λ 1 is the smallest eigenvalue of A) This is called inverse iteration 1: choose x 0 C n arbitrarily 2: for k = 1, 2,... do 3: solve Ay k = x k 1 for y k 4: x k = y k / y k 5: end for
29 Inverse Iteration Hence inverse iteration gives λ 1 without requiring a shift This is helpful since it may be difficult to determine what shift is required to get λ 1 in the power method Question: What happens if we apply inverse iteration to the shifted matrix A σi?
30 Inverse Iteration The smallest eigenvalue of A σi is (λ i σ), where i = arg min λ i σ, i=1,2,...,n and hence... Answer: We converge to λ = 1/(λ i σ), then recover λ i via λ i = 1 λ + σ Inverse iteration with shift allows us to find the eigenvalue closest to σ
31 Rayleigh Quotient Iteration
32 Rayleigh Quotient For the remainder of this section (Rayleigh Quotient Iteration, QR Algorithm) we will assume that A R n n is real and symmetric 3 The Rayleigh quotient is defined as r(x) x T Ax x T x If (λ, v) R R n is an eigenpair, then r(v) = v T Av v T v = λv T v v T v = λ 3 Much of the material generalizes to complex nonhermitian matrices, but symmetric case is simpler
33 Rayleigh Quotient Theorem: Suppose A R n n is a symmetric matrix, then for any x R n we have λ 1 r(x) λ n Proof: We write x as a linear combination of (orthogonal) eigenvectors x = n j=1 α jv j, and the lower bound follows from r(x) = x T n Ax x T x = j=1 λ jαj 2 n j=1 α2 j n j=1 λ α2 j 1 n j=1 α2 j = λ 1 The proof of the upper bound r(x) λ n is analogous
34 Rayleigh Quotient Corollary: A symmetric matrix A R n n is positive definite if and only if all of its eigenvalues are positive Proof: ( ) Suppose A is symmetric positive definite (SPD), then for any nonzero x R n, we have x T Ax > 0 and hence λ 1 = r(v 1 ) = v T 1 Av 1 v T 1 v 1 > 0 ( ) Suppose A has positive eigenvalues, then for any nonzero x R n x T Ax = r(x)(x T x) λ 1 x 2 2 > 0
35 Rayleigh Quotient But also, if x is an approximate eigenvector, then r(x) gives us a good approximation to the eigenvalue This is because estimation of an eigenvalue from an approximate eigenvector is an n 1 linear least squares problem: xλ Ax x R n is our tall thin matrix and Ax R n is our righthand side Hence the normal equation for xλ Ax yields the Rayleigh quotient, i.e. x T xλ = x T Ax
36 Rayleigh Quotient Question: How accurate is the Rayleigh quotient approximation to an eigenvalue? Let s consider r as a function of x, so r : R n R r(x) x j = x j (x T Ax) x T x (x T Ax) x j (x T x) (x T x) 2 = 2(Ax) j x T (x T Ax)2x j x (x T x) 2 2 = x T x (Ax r(x)x) j (Note that the second equation relies on the symmetry of A)
37 Rayleigh Quotient Therefore r(x) = 2 x T (Ax r(x)x) x For an eigenpair (λ, v) we have r(v) = λ and hence r(v) = 2 v T (Av λv) = 0 v This shows that eigenvectors of A are stationary points of r
38 Rayleigh Quotient Suppose (λ, v) is an eigenpair of A, and let us consider a Taylor expansion of r(x) about v: r(x) = r(v) + r(v) T (x v) (x v)t H r (v)(x v) + H.O.T. = r(v) (x v)t H r (v)(x v) + H.O.T. Hence as x v the error in a Rayleigh quotient approximation is r(x) λ = O( x v 2 2) That is, the Rayleigh quotient approx. to an eigenvalue squares the error in a corresponding eigenvector approx.
39 Rayleigh Quotient Iteration The Rayleigh quotient gives us an eigenvalue estimate from an eigenvector estimate Inverse iteration gives us an eigenvector estimate from an eigenvalue estimate It is natural to combine the two, this yields the Rayleigh quotient iteration 1: choose x 0 R n arbitrarily 2: for k = 1, 2,... do 3: σ k = x T k 1 Ax k 1/x T k 1 x k 1 4: solve (A σ k I)y k = x k 1 for y k 5: x k = y k / y k 6: end for
40 Rayleigh Quotient Iteration Suppose, at step k, we have x k 1 v ɛ Then, from the Rayleigh quotient in line 3 of the algorithm, we have σ k λ = O(ɛ 2 ) In lines 4 and 5 of the algorithm, we then perform an inverse iteration with shift σ k to get x k Recall the eigenvector error in one inverse iteration step is scaled by ratio of second largest to largest eigenvalues of (A σ k I) 1
41 Rayleigh Quotient Iteration Let λ be the closest eigenvalue of A to σ k, then the magnitude of largest eigenvalue of (A σ k I) 1 is 1/ σ k λ The second largest eigenvalue magnitude is 1/ σ k ˆλ, where ˆλ is the eigenvalue of A second closest to σ k Hence at each inverse iteration step, the error is reduced by a factor σ k λ σ k ˆλ = σ k λ (σ k λ) + (λ ˆλ) const. σ k λ as σ k λ Therefore, we obtain cubic convergence as k : x k v (const. σ k λ ) x k 1 v = O(ɛ 3 )
42 Rayleigh Quotient Iteration A drawback of Rayleigh iteration: we can t just LU factorize A σ k I once since the shift changes each step Also, it s harder to pick out specific parts of the spectrum with Rayleigh quotient iteration since σ k can change unpredictably Python demo: Rayleigh iteration to compute an eigenpair of A =
43 QR Algorithm
44 The QR Algorithm The QR algorithm for computing eigenvalues is one of the best known algorithms in Numerical Analysis 4 It was developed independently in the late 1950s by John G.F. Francis (England) and Vera N. Kublanovskaya (USSR) The QR algorithm efficiently provides approximations for all eigenvalues/eigenvectors of a matrix We will consider what happens when we apply the power method to a set of vectors this will then motivate the QR algorithm 4 Recall that here we focus on the case in which A R n n is symmetric
45 The QR Algorithm Let x (0) 1,..., x p (0) denote p linearly independent starting vectors, and suppose we store these vectors in the columns of X 0 We can apply the power method to these vectors to obtain the following algorithm: 1: choose an n p matrix X 0 arbitrarily 2: for k = 1, 2,... do 3: X k = AX k 1 4: end for
46 The QR Algorithm From our analysis of the power method, we see that for each i = 1, 2,..., p: ) x (k) i = (λ k nα i,n v n + λ k n 1α i,n 1 v n λ k 1α i,1 v 1 n ( ) k n p = λ k λj ( ) k λj n p α i,jv j + α i,jv j j=n p+1 λ n p j=1 λ n p Then, if λ n p+1 > λ n p, the sum in green will decay compared to the sum in blue as k Hence the columns of X k will converge to a basis for span{v n p+1,..., v n }
47 The QR Algorithm However, this method doesn t provide a good basis: each column of X k will be very close to v n Therefore the columns of X k become very close to being linearly dependent We can resolve this issue by enforcing linear independence at each step
48 The QR Algorithm We orthonormalize the vectors after each iteration via a (reduced) QR factorization, to obtain the simultaneous iteration: 1: choose n p matrix Q 0 with orthonormal columns 2: for k = 1, 2,... do 3: X k = A ˆQ k 1 4: ˆQ k ˆR k = X k 5: end for The column spaces of ˆQ k and X k in line 4 are the same Hence columns of ˆQ k converge to orthonormal basis for span{v n p+1,..., v n }
49 The QR Algorithm In fact, we don t just get a basis for span{v n p+1,..., v n }, we get the eigenvectors themselves! Theorem: The columns of ˆQ k converge to the p dominant eigenvectors of A We will not discuss the full proof, but we note that this result is not surprising since: the eigenvectors of a symmetric matrix are orthogonal columns of ˆQ k converge to an orthogonal basis for span{v n p+1,..., v n } Simultaneous iteration approximates eigenvectors, we obtain eigenvalues from the Rayleigh quotient ˆQ T A ˆQ diag(λ 1,..., λ n )
50 The QR Algorithm With p = n, the simultaneous iteration will approximate all eigenpairs of A We now show a more convenient reorganization of the simultaneous iteration algorithm We shall require some extra notation: the Q and R matrices arising in the simultaneous iteration will be underlined Q k, R k (As we will see shortly, this is to distinguish between the matrices arising in the two different formulations...)
Applied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.2: Fundamentals 2 / 31 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors of
More informationEIGENVALUE PROBLEMS. Background on eigenvalues/ eigenvectors / decompositions. Perturbation analysis, condition numbers..
EIGENVALUE PROBLEMS Background on eigenvalues/ eigenvectors / decompositions Perturbation analysis, condition numbers.. Power method The QR algorithm Practical QR algorithms: use of Hessenberg form and
More informationEigenvalue and Eigenvector Problems
Eigenvalue and Eigenvector Problems An attempt to introduce eigenproblems Radu Trîmbiţaş BabeşBolyai University April 8, 2009 Radu Trîmbiţaş ( BabeşBolyai University) Eigenvalue and Eigenvector Problems
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Eigenvalue Problems; Similarity Transformations Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 18 Eigenvalue
More informationLast Time. Social Network Graphs Betweenness. Graph Laplacian. GirvanNewman Algorithm. Spectral Bisection
Eigenvalue Problems Last Time Social Network Graphs Betweenness GirvanNewman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Power iteration Notes for 20161017 In most introductory linear algebra classes, one computes eigenvalues as roots of a characteristic polynomial. For most problems, this is a bad idea: the roots of
More informationMath 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm
Math 405: Numerical Methods for Differential Equations 2016 W1 Topics 10: Matrix Eigenvalues and the Symmetric QR Algorithm References: Trefethen & Bau textbook Eigenvalue problem: given a matrix A, find
More informationLecture 13 Eigenvalue Problems
Lecture 13 Eigenvalue Problems MIT 18.335J / 6.337J Introduction to Numerical Methods PerOlof Persson (persson@mit.edu) October 24, 2007 1 The Eigenvalue Decomposition Eigenvalue problem for m m matrix
More informationEigenvalues and Eigenvectors
Chapter 1 Eigenvalues and Eigenvectors Among problems in numerical linear algebra, the determination of the eigenvalues and eigenvectors of matrices is second in importance only to the solution of linear
More informationEigenvalue Problems. Eigenvalue problems occur in many areas of science and engineering, such as structural analysis
Eigenvalue Problems Eigenvalue problems occur in many areas of science and engineering, such as structural analysis Eigenvalues also important in analyzing numerical methods Theory and algorithms apply
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 4 Eigenvalue Problems Prof. Michael T. Heath Department of Computer Science University of Illinois at UrbanaChampaign Copyright c 2002. Reproduction
More informationApplied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.4: Krylov Subspace Methods 2 / 51 Krylov Subspace Methods In this chapter we give
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Solving Eigenvalue Problems All
More informationThe Eigenvalue Problem: Perturbation Theory
Jim Lambers MAT 610 Summer Session 200910 Lecture 13 Notes These notes correspond to Sections 7.2 and 8.1 in the text. The Eigenvalue Problem: Perturbation Theory The Unsymmetric Eigenvalue Problem Just
More informationNotes on Eigenvalues, Singular Values and QR
Notes on Eigenvalues, Singular Values and QR Michael Overton, Numerical Computing, Spring 2017 March 30, 2017 1 Eigenvalues Everyone who has studied linear algebra knows the definition: given a square
More information33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM
33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM (UPDATED MARCH 17, 2018) The final exam will be cumulative, with a bit more weight on more recent material. This outline covers the what we ve done since the
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 21: Sensitivity of Eigenvalues and Eigenvectors; Conjugate Gradient Method Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis
More informationMATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2.
MATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2. Diagonalization Let L be a linear operator on a finitedimensional vector space V. Then the following conditions are equivalent:
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J Olver 8 Numerical Computation of Eigenvalues In this part, we discuss some practical methods for computing eigenvalues and eigenvectors of matrices Needless to
More informationAnnouncements Monday, November 06
Announcements Monday, November 06 This week s quiz: covers Sections 5 and 52 Midterm 3, on November 7th (next Friday) Exam covers: Sections 3,32,5,52,53 and 55 Section 53 Diagonalization Motivation: Difference
More informationMATH 304 Linear Algebra Lecture 33: Bases of eigenvectors. Diagonalization.
MATH 304 Linear Algebra Lecture 33: Bases of eigenvectors. Diagonalization. Eigenvalues and eigenvectors of an operator Definition. Let V be a vector space and L : V V be a linear operator. A number λ
More informationarxiv: v1 [math.na] 5 May 2011
ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU arxiv:1105.1185v1 [math.na] 5 May 2011 Abstract. We examine some numerical iterative methods for computing the eigenvalues and
More informationLecture 4 Eigenvalue problems
Lecture 4 Eigenvalue problems Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn
More informationDIAGONALIZATION. In order to see the implications of this definition, let us consider the following example Example 1. Consider the matrix
DIAGONALIZATION Definition We say that a matrix A of size n n is diagonalizable if there is a basis of R n consisting of eigenvectors of A ie if there are n linearly independent vectors v v n such that
More informationLecture 15, 16: Diagonalization
Lecture 15, 16: Diagonalization Motivation: Eigenvalues and Eigenvectors are easy to compute for diagonal matrices. Hence, we would like (if possible) to convert matrix A into a diagonal matrix. Suppose
More informationEIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4
EIGENVALUE PROBLEMS EIGENVALUE PROBLEMS p. 1/4 EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue
More information1. In this problem, if the statement is always true, circle T; otherwise, circle F.
Math 1553, Extra Practice for Midterm 3 (sections 4565) Solutions 1 In this problem, if the statement is always true, circle T; otherwise, circle F a) T F If A is a square matrix and the homogeneous equation
More informationApplied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.3: QR Factorization, SVD 2 / 66 QR Factorization 3 / 66 QR Factorization
More informationThroughout these notes we assume V, W are finite dimensional inner product spaces over C.
Math 342  Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal
More informationEigenproblems II: Computation
Eigenproblems II: Computation CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Eigenproblems II: Computation 1 / 31 Setup A
More informationNumerical Methods I Eigenvalue Problems
Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATHGA 2011.003 / CSCIGA 2945.003, Fall 2014 October 2nd, 2014 A. Donev (Courant Institute) Lecture
More informationComputational Methods CMSC/AMSC/MAPL 460. Eigenvalues and Eigenvectors. Ramani Duraiswami, Dept. of Computer Science
Computational Methods CMSC/AMSC/MAPL 460 Eigenvalues and Eigenvectors Ramani Duraiswami, Dept. of Computer Science Eigen Values of a Matrix Recap: A N N matrix A has an eigenvector x (nonzero) with corresponding
More informationComputation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated.
Math 504, Homework 5 Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated 1 Find the eigenvalues and the associated eigenspaces
More informationEigenvalues and eigenvectors
Chapter 6 Eigenvalues and eigenvectors An eigenvalue of a square matrix represents the linear operator as a scaling of the associated eigenvector, and the action of certain matrices on general vectors
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Reduction to Hessenberg and Tridiagonal Forms; Rayleigh Quotient Iteration Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationMATH 304 Linear Algebra Lecture 20: The GramSchmidt process (continued). Eigenvalues and eigenvectors.
MATH 304 Linear Algebra Lecture 20: The GramSchmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v
More information22.4. Numerical Determination of Eigenvalues and Eigenvectors. Introduction. Prerequisites. Learning Outcomes
Numerical Determination of Eigenvalues and Eigenvectors 22.4 Introduction In Section 22. it was shown how to obtain eigenvalues and eigenvectors for low order matrices, 2 2 and. This involved firstly solving
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More information18.06 Problem Set 8 Solution Due Wednesday, 22 April 2009 at 4 pm in Total: 160 points.
86 Problem Set 8 Solution Due Wednesday, April 9 at 4 pm in 6 Total: 6 points Problem : If A is realsymmetric, it has real eigenvalues What can you say about the eigenvalues if A is real and antisymmetric
More informationAnnouncements Wednesday, November 7
Announcements Wednesday, November 7 The third midterm is on Friday, November 6 That is one week from this Friday The exam covers 45, 5, 52 53, 6, 62, 64, 65 (through today s material) WeBWorK 6, 62 are
More informationEigenvalues, Eigenvectors, and Diagonalization
Week12 Eigenvalues, Eigenvectors, and Diagonalization 12.1 Opening Remarks 12.1.1 Predicting the Weather, Again Let us revisit the example from Week 4, in which we had a simple model for predicting the
More informationEigenvalues, Eigenvectors, and Diagonalization
Week12 Eigenvalues, Eigenvectors, and Diagonalization 12.1 Opening Remarks 12.1.1 Predicting the Weather, Again Let us revisit the example from Week 4, in which we had a simple model for predicting the
More informationMath 471 (Numerical methods) Chapter 4. Eigenvalues and Eigenvectors
Math 471 (Numerical methods) Chapter 4. Eigenvalues and Eigenvectors Overlap 4.1 4.3 of Bradie 4.0 Review on eigenvalues and eigenvectors Definition. A nonzero vector v is called an eigenvector of A if
More informationLinear algebra & Numerical Analysis
Linear algebra & Numerical Analysis Eigenvalues and Eigenvectors Marta Jarošová http://homel.vsb.cz/~dom033/ Outline Methods computing all eigenvalues Characteristic polynomial Jacobi method for symmetric
More informationMath 240 Calculus III
Generalized Calculus III Summer 2015, Session II Thursday, July 23, 2015 Agenda 1. 2. 3. 4. Motivation Defective matrices cannot be diagonalized because they do not possess enough eigenvectors to make
More informationLecture 10  Eigenvalues problem
Lecture 10  Eigenvalues problem Department of Computer Science University of Houston February 28, 2008 1 Lecture 10  Eigenvalues problem Introduction Eigenvalue problems form an important class of problems
More informationLecture 8 : Eigenvalues and Eigenvectors
CPS290: Algorithmic Foundations of Data Science February 24, 2017 Lecture 8 : Eigenvalues and Eigenvectors Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Hermitian Matrices It is simpler to begin with
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More information1 Last time: leastsquares problems
MATH Linear algebra (Fall 07) Lecture Last time: leastsquares problems Definition. If A is an m n matrix and b R m, then a leastsquares solution to the linear system Ax = b is a vector x R n such that
More informationEigenvalues, Eigenvectors, and Diagonalization
Week12 Eigenvalues, Eigenvectors, and Diagonalization 12.1 Opening Remarks 12.1.1 Predicting the Weather, Again View at edx Let us revisit the example from Week 4, in which we had a simple model for predicting
More informationMATH 583A REVIEW SESSION #1
MATH 583A REVIEW SESSION #1 BOJAN DURICKOVIC 1. Vector Spaces Very quick review of the basic linear algebra concepts (see any linear algebra textbook): (finite dimensional) vector space (or linear space),
More information4.2. ORTHOGONALITY 161
4.2. ORTHOGONALITY 161 Definition 4.2.9 An affine space (E, E ) is a Euclidean affine space iff its underlying vector space E is a Euclidean vector space. Given any two points a, b E, we define the distance
More informationAM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition
AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized
More informationAnnouncements Wednesday, November 7
Announcements Wednesday, November 7 The third midterm is on Friday, November 16 That is one week from this Friday The exam covers 45, 51, 52 53, 61, 62, 64, 65 (through today s material) WeBWorK 61, 62
More informationMATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.
MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation
More informationAnnouncements Monday, November 13
Announcements Monday, November 13 The third midterm is on this Friday, November 17 The exam covers 31, 32, 51, 52, 53, and 55 About half the problems will be conceptual, and the other half computational
More informationftuiowamath2550 Assignment NOTRequiredJustHWformatOfQuizReviewForExam3part2 due 12/31/2014 at 07:10pm CST
me me ftuiowamath2550 Assignment NOTRequiredJustHWformatOfQuizReviewForExam3part2 due 12/31/2014 at 07:10pm CST 1. (1 pt) local/library/ui/eigentf.pg A is n n an matrices.. There are an infinite number
More informationLinear Algebra Exercises
9. 8.03 Linear Algebra Exercises 9A. Matrix Multiplication, Rank, Echelon Form 9A. Which of the following matrices is in rowechelon form? 2 0 0 5 0 (i) (ii) (iii) (iv) 0 0 0 (v) [ 0 ] 0 0 0 0 0 0 0 9A2.
More informationLU Factorization. LU factorization is the most common way of solving linear systems! Ax = b LUx = b
AM 205: lecture 7 Last time: LU factorization Today s lecture: Cholesky factorization, timing, QR factorization Reminder: assignment 1 due at 5 PM on Friday September 22 LU Factorization LU factorization
More information235 Final exam review questions
5 Final exam review questions Paul Hacking December 4, 0 () Let A be an n n matrix and T : R n R n, T (x) = Ax the linear transformation with matrix A. What does it mean to say that a vector v R n is an
More informationLinear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a ToolBox Linear Equation Systems Discretization of differential equations: solving linear equations
More information2 Eigenvectors and Eigenvalues in abstract spaces.
MA322 Sathaye Notes on Eigenvalues Spring 27 Introduction In these notes, we start with the definition of eigenvectors in abstract vector spaces and follow with the more common definition of eigenvectors
More informationChapter 5. Eigenvalues and Eigenvectors
Chapter 5 Eigenvalues and Eigenvectors Section 5. Eigenvectors and Eigenvalues Motivation: Difference equations A Biology Question How to predict a population of rabbits with given dynamics:. half of the
More informationQuadratic forms. Here. Thus symmetric matrices are diagonalizable, and the diagonalization can be performed by means of an orthogonal matrix.
Quadratic forms 1. Symmetric matrices An n n matrix (a ij ) n ij=1 with entries on R is called symmetric if A T, that is, if a ij = a ji for all 1 i, j n. We denote by S n (R) the set of all n n symmetric
More informationAM 205: lecture 6. Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization
AM 205: lecture 6 Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization Unit II: Numerical Linear Algebra Motivation Almost everything in Scientific Computing
More informationEECS 275 Matrix Computation
EECS 275 Matrix Computation MingHsuan Yang Electrical Engineering and Computer Science University of California at Merced Merced, CA 95344 http://faculty.ucmerced.edu/mhyang Lecture 17 1 / 26 Overview
More informationDSGA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DSGA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationChapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015
Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal
More informationMATH 304 Linear Algebra Lecture 34: Review for Test 2.
MATH 304 Linear Algebra Lecture 34: Review for Test 2. Topics for Test 2 Linear transformations (Leon 4.1 4.3) Matrix transformations Matrix of a linear mapping Similar matrices Orthogonality (Leon 5.1
More informationMA 265 FINAL EXAM Fall 2012
MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators
More informationAM 205: lecture 6. Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization
AM 205: lecture 6 Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization Unit II: Numerical Linear Algebra Motivation Almost everything in Scientific Computing
More informationMath 110 Linear Algebra Midterm 2 Review October 28, 2017
Math 11 Linear Algebra Midterm Review October 8, 17 Material Material covered on the midterm includes: All lectures from Thursday, Sept. 1st to Tuesday, Oct. 4th Homeworks 9 to 17 Quizzes 5 to 9 Sections
More informationThe QR Decomposition
The QR Decomposition We have seen one major decomposition of a matrix which is A = LU (and its variants) or more generally PA = LU for a permutation matrix P. This was valid for a square matrix and aided
More informationMath 3191 Applied Linear Algebra
Math 9 Applied Linear Algebra Lecture 9: Diagonalization Stephen Billups University of Colorado at Denver Math 9Applied Linear Algebra p./9 Section. Diagonalization The goal here is to develop a useful
More informationCheat Sheet for MATH461
Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A
More informationJim Lambers MAT 610 Summer Session Lecture 2 Notes
Jim Lambers MAT 610 Summer Session 200910 Lecture 2 Notes These notes correspond to Sections 2.22.4 in the text. Vector Norms Given vectors x and y of length one, which are simply scalars x and y, the
More informationChapter 7. Canonical Forms. 7.1 Eigenvalues and Eigenvectors
Chapter 7 Canonical Forms 7.1 Eigenvalues and Eigenvectors Definition 7.1.1. Let V be a vector space over the field F and let T be a linear operator on V. An eigenvalue of T is a scalar λ F such that there
More informationTHE MATRIX EIGENVALUE PROBLEM
THE MATRIX EIGENVALUE PROBLEM Find scalars λ and vectors x 0forwhich Ax = λx The form of the matrix affects the way in which we solve this problem, and we also have variety as to what is to be found. A
More informationJordan Canonical Form Homework Solutions
Jordan Canonical Form Homework Solutions For each of the following, put the matrix in Jordan canonical form and find the matrix S such that S AS = J. [ ]. A = A λi = λ λ = ( λ) = λ λ = λ =, Since we have
More informationNumerical Solution of Linear Eigenvalue Problems
Numerical Solution of Linear Eigenvalue Problems Jessica Bosch and Chen Greif Abstract We review numerical methods for computing eigenvalues of matrices We start by considering the computation of the dominant
More informationComputing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices
Computing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 9, 2008 Today
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University October 17, 005 Lecture 3 3 he Singular Value Decomposition
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationMath 4A Notes. Written by Victoria Kala Last updated June 11, 2017
Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...
More informationIn this section again we shall assume that the matrix A is m m, real and symmetric.
84 3. The QR algorithm without shifts See Chapter 28 of the textbook In this section again we shall assume that the matrix A is m m, real and symmetric. 3.1. Simultaneous Iterations algorithm Suppose we
More informationLinear Algebra Review
Chapter 1 Linear Algebra Review It is assumed that you have had a course in linear algebra, and are familiar with matrix multiplication, eigenvectors, etc. I will review some of these terms here, but quite
More informationMIDTERM. b) [2 points] Compute the LU Decomposition A = LU or explain why one does not exist.
MAE 9A / FALL 3 Maurício de Oliveira MIDTERM Instructions: You have 75 minutes This exam is open notes, books No computers, calculators, phones, etc There are 3 questions for a total of 45 points and bonus
More informationComputing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices
Computing Eigenvalues and/or Eigenvectors;Part 1, Generalities and symmetric matrices Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 8, 2009 Today
More informationj=1 u 1jv 1j. 1/ 2 Lemma 1. An orthogonal set of vectors must be linearly independent.
Lecture Notes: Orthogonal and Symmetric Matrices Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Orthogonal Matrix Definition. Let u = [u
More informationLinear Algebra: Matrix Eigenvalue Problems
CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given
More informationLinear Algebra. Carleton DeTar February 27, 2017
Linear Algebra Carleton DeTar detar@physics.utah.edu February 27, 2017 This document provides some background for various course topics in linear algebra: solving linear systems, determinants, and finding
More informationOn the Modification of an Eigenvalue Problem that Preserves an Eigenspace
Purdue University Purdue epubs Department of Computer Science Technical Reports Department of Computer Science 2009 On the Modification of an Eigenvalue Problem that Preserves an Eigenspace Maxim Maumov
More informationMath 205, Summer I, Week 4b:
Math 205, Summer I, 2016 Week 4b: Chapter 5, Sections 6, 7 and 8 (5.5 is NOT on the syllabus) 5.6 Eigenvalues and Eigenvectors 5.7 Eigenspaces, nondefective matrices 5.8 Diagonalization [*** See next slide
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More informationECS231 Handout Subspace projection methods for Solving LargeScale Eigenvalue Problems. Part I: Review of basic theory of eigenvalue problems
ECS231 Handout Subspace projection methods for Solving LargeScale Eigenvalue Problems Part I: Review of basic theory of eigenvalue problems 1. Let A C n n. (a) A scalar λ is an eigenvalue of an n n A
More informationEigenvalues, Eigenvectors, and Diagonalization
Math 240 TA: Shuyi Weng Winter 207 February 23, 207 Eigenvalues, Eigenvectors, and Diagonalization The concepts of eigenvalues, eigenvectors, and diagonalization are best studied with examples. We will
More informationReview of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem
Review of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem Steven J. Miller June 19, 2004 Abstract Matrices can be thought of as rectangular (often square) arrays of numbers, or as
More informationBASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x
BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,
More information