AMS526: Numerical Analysis (Numerical Linear Algebra for Computational and Data Sciences) Lecture 14: Eigenvalue Problems; Eigenvalue Revealing Factorizations Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis 1 / 14
Outline 1 Properties of Eigenvalue Problems (NLA 24) 2 Eigenvalue Revealing Factorizations (NLA 24) Xiangmin Jiao Numerical Analysis 2 / 14
Eigenvalue and Eigenvectors Eigenvalue problem of n n matrix A is Ax = x with eigenvalues and eigenvectors x (nonzero) The set of all the eigenvalues of A is the spectrum of A Eigenvalue are generally used where a matrix is to be compounded iteratively Eigenvalues are useful for algorithmic and physical reasons Algorithmically, eigenvalue analysis can reduce a coupled system to a collection of scalar problems Physically, eigenvalue analysis can be used to study resonance of musical instruments and stability of physical systems Xiangmin Jiao Numerical Analysis 3 / 14
Eigenvalue Decomposition Eigenvalue decomposition of A is A = X X 1 or AX = X with eigenvectors x i as columns of X and eigenvalues diagonal of. Alternatively, i along Ax i = i x i Eigenvalue decomposition is change of basis to eigenvector coordinates Ax = b! (X 1 b)= (X 1 x) Note that eigenvalue decomposition may not exist Question: How does eigenvalue decomposition differ from SVD? Xiangmin Jiao Numerical Analysis 4 / 14
Geometric Multiplicity Eigenvectors corresponding to a single eigenvalue form an eigenspace E C n n Eigenspace is invariant in that AE E Dimension of E is the maximum number of linearly independent eigenvectors that can be found Geometric multiplicity of is dimension of E, i.e., dim(null(a )) Xiangmin Jiao Numerical Analysis 5 / 14
Algebraic Multiplicity The characteristic polynomial of A is degree m polynomial p A (z) =det(z A) =(z 1)(z 2) (z n) which is monic in that coefficient of z n is 1 is eigenvalue of A iff p A ( )=0 f is eigenvalue, then by definition, x Ax =( A)x = 0, so ( A) is singular and its determinant is 0 f ( A) is singular, then for x 2 null( A) we have x Ax = 0 Algebraic multiplicity of is its multiplicity as a root of p A Any matrix A 2 C n n has n eigenvalues, counted with algebraic multiplicity Question: What are the eigenvalues of a triangular matrix? Question: How are geometric multiplicity and algebraic multiplicity related? Xiangmin Jiao Numerical Analysis 6 / 14
Similarity Transformations The map A! Y 1 AY is a similarity transformation of A for any nonsingular Y 2 C n n A and B are similar if there is a similarity transformation B = Y 1 AY Theorem f Y is nonsingular, then A and Y 1 AY have the same characteristic polynomials, eigenvalues, and algebraic and geometric multiplicities. 1 For characteristic polynomial: det(z Y 1 AY )=det(y 1 (z A)Y )=det(z A) so algebraic multiplicities remain the same 2 f x 2 E for A, theny 1 x is in eigenspace of Y 1 AY corresponding to,andviceversa,sogeometricmultiplicitiesremainthesame Xiangmin Jiao Numerical Analysis 7 / 14
Algebraic Multiplicity Geometric Multiplicity Let k be be geometric multiplicity of for A. LetˆV 2 C n k constitute of orthonormal basis of the E Extend ˆV to unitary V [ ˆV, Ṽ ] 2 C n n and form apple ˆV B = V AV = A ˆV ˆV apple AṼ C Ṽ A ˆV Ṽ = AṼ 0 D det(z B) =det(z )det(z D) =(z ) k det(z D), sothe algebraic multiplicity of as an eigenvalue of B is k A and B are similar, so the algebraic multiplicity of as an eigenvalue of A is at least k Examples: 2 3 2 3 2 2 1 A = 4 2 5, B = 4 2 1 5 2 2 Their characteristic polynomial is (z 2) 3,soalgebraicmultiplicityof = 2is3.ButgeometricmultiplicityofA is 3 and that of B is 1. Xiangmin Jiao Numerical Analysis 8 / 14
Defective and Diagonalizable Matrices An eigenvalue of a matrix is defective if its algebraic multiplicity > its geometric multiplicity A matrix is defective if it has a defective eigenvalue. Otherwise, it is called nondefective. Theorem An n n matrix A is nondefective iff it has an eigenvalue decomposition A = X X 1. Xiangmin Jiao Numerical Analysis 9 / 14
Defective and Diagonalizable Matrices An eigenvalue of a matrix is defective if its algebraic multiplicity > its geometric multiplicity A matrix is defective if it has a defective eigenvalue. Otherwise, it is called nondefective. Theorem An n n matrix A is nondefective iff it has an eigenvalue decomposition A = X X 1. (() is nondefective, and A is similar to, soa is nondefective. ()) A nondefective matrix has n linearly independent eigenvectors. Take them as columns of X to obtain A = X X 1. Nondefective matrices are therefore also said to be diagonalizable. Xiangmin Jiao Numerical Analysis 9 / 14
Determinant and Trace Determinant of A is det(a) = Q n j=1 j, because det(a) =( 1) n det( A) =( 1) n p A (0) = ny j=1 j Trace of A is tr(a) = P n j=1 j, since nx p A (z) =det(z A) =z n a jj z n 1 + O(z n 2 ) p A (z) = j=1 ny nx (z j) =z n j=1 j=1 jz n 1 + O(z n 2 ) Question: Are these results valid for defective or nondefective matrices? Xiangmin Jiao Numerical Analysis 10 / 14
Outline 1 Properties of Eigenvalue Problems (NLA 24) 2 Eigenvalue Revealing Factorizations (NLA 24) Xiangmin Jiao Numerical Analysis 11 / 14
Unitary Diagonalization AmatrixA is unitarily diagonalizable if A = Q Q for a unitary matrix Q AHermitianmatrixisunitarilydiagonalizable,withrealeigenvalues AmatrixA is normal if A A = AA Examples of normal matrices include Hermitian matrices, skew Hermitian matrices Hermitian, matrix is normal and all eigenvalues are real skew Hermitian, matrix is normal and all eigenvalues are imaginary f A is both triangular and normal, then A is diagonal Unitarily diagonalizable, normal ) is easy. Prove ( by induction using Schur factorization next Xiangmin Jiao Numerical Analysis 12 / 14
Schur Factorization Schur factorization is A = QTQ,whereQ is unitary and T is upper triangular Theorem Every square matrix A has a Schur factorization. Proof by induction on dimension of A. Casen = 1istrivial. For n 2, let x be any unit eigenvector of A, withcorresponding eigenvalue.letube unitary matrix with x as first column. Then apple U w AU =. 0 C By induction hypothesis, there is a Schur factorization T = V CV.Let apple apple 1 0 Q = U 0 V, T = w V, 0 T and then A = QTQ. Xiangmin Jiao Numerical Analysis 13 / 14
Eigenvalue Revealing Factorizations Eigenvalue-revealing factorization of square matrix A Diagonalization A = X X 1 (nondefective A) Unitary Diagonalization A = Q Q (normal A) Unitary triangularization (Schur factorization) A = QTQ (any A) Jordan 2 normal form A = XJX 3 1,whereJ block diagonal with J i = 6 4 i 1 i...... 1 i 7 5 n general, Schur factorization is used, because Unitary matrices are involved, so algorithm tends to be more stable f A is normal, then Schur form is diagonal Xiangmin Jiao Numerical Analysis 14 / 14