Lecture 11: Finish Gaussian elimination and applications; intro to eigenvalues and eigenvectors (1) Travis Schedler Tue, Oct 18, 2011 (version: Tue, Oct 18, 6:00 PM)
Goals (2) Solving systems of equations PLU decomposition of matrices Eigenvalues and eigenvectors Invariant subspaces Read Gaussian Elimination handout (on website).
Warm-up exercise (3) Let V be a f.d. vs. Let T L(V ). Read both of the following, then prove the one your group is assigned to: (a) Prove that (I, T, T 2,..., T (dim V )2 ) is linearly dependent. Hint: Use that dim L(V ) = (dim V ) 2 (we proved this from the fact that M is an isomorphism). (b) Recall from the PS that, if p(x) = a m x m + + a 1 x + a 0 is a polynomial, then we say p(t ) = a m T m + + a 1 T + a 0 I. Using (a), prove that, for some nonzero polynomial p of degree at most (dim V ) 2, p(t ) = 0. Now, if F = C, we can write p(x) = a m (x r 1 ) (x r m ) (fundamental theorem of algebra). From the last PS (#9), if V {0}, we conclude that, for some j, (T r j I ) is not injective. That is, there exists a nonzero v null(t r j I ), i.e., a nonzero eigenvector v of eigenvalue r j C (Tv = r j v).
Solution to warm-up exercise (4) (a) Since this is a list of length > (dim V ) 2 in a vector space of dimension (dim V ) 2, it must be linearly dependent.
Solution to warm-up exercise (4) (a) Since this is a list of length > (dim V ) 2 in a vector space of dimension (dim V ) 2, it must be linearly dependent. (b) From (a), a 0 I + a 1 T + + a (dim V ) 2T (dim V )2 = 0 for some a 0,..., a (dim V ) 2. Set p(x) = a 0 + a 1 x + + a (dim V ) 2x (dim V )2. Then p(t ) = 0.
Solving systems of equations (5) Observe: A system of linear equations i,j a i,jx j = b i is Ax = b, A Mat(m, n, F), b Mat(m, 1, F), x =.. To solve: Perform G. or G-J elim. on (A b) (on board). x 1 x n
Solving systems of equations (5) Observe: A system of linear equations i,j a i,jx j = b i is Ax = b, A Mat(m, n, F), b Mat(m, 1, F), x =.. To solve: Perform G. or G-J elim. on (A b) (on board). Equiv: If SA = C, for C (reduced) row echelon, then Ax = b is equiv. to Cx = Sb. Here we just set free entries of x arbitrarily and solve for pivot entries. x 1 x n
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B. The E i that are not permutations are lower triangular.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B. The E i that are not permutations are lower triangular. The product of lower-tri matrices is also lower-tri.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B. The E i that are not permutations are lower triangular. The product of lower-tri matrices is also lower-tri. So if there are no permutations, we get SA = B, S is lower tri. So A = LB, for L = S 1 also lower tri.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B. The E i that are not permutations are lower triangular. The product of lower-tri matrices is also lower-tri. So if there are no permutations, we get SA = B, S is lower tri. So A = LB, for L = S 1 also lower tri. If there are permutations, we can do those first, once we know what they are.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B. The E i that are not permutations are lower triangular. The product of lower-tri matrices is also lower-tri. So if there are no permutations, we get SA = B, S is lower tri. So A = LB, for L = S 1 also lower tri. If there are permutations, we can do those first, once we know what they are. Then, P 1 A = LB for L lower-tri, and P 1 a permutation matrix.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B. The E i that are not permutations are lower triangular. The product of lower-tri matrices is also lower-tri. So if there are no permutations, we get SA = B, S is lower tri. So A = LB, for L = S 1 also lower tri. If there are permutations, we can do those first, once we know what they are. Then, P 1 A = LB for L lower-tri, and P 1 a permutation matrix. Upshot: A = PLB.
PLU decomposition (6) Motivation: View SA = B, for B row echelon, as a decomposition, A = S 1 B. Gaussian elimination gives E m E m 1 E 1 A = B. The E i that are not permutations are lower triangular. The product of lower-tri matrices is also lower-tri. So if there are no permutations, we get SA = B, S is lower tri. So A = LB, for L = S 1 also lower tri. If there are permutations, we can do those first, once we know what they are. Then, P 1 A = LB for L lower-tri, and P 1 a permutation matrix. Upshot: A = PLB. If B is invertible [which requires B to be square, i.e., m = n], then it is upper-triangular. Then, write U = B, and A = PLU.
Uniqueness of reduced row echelon form (7) Theorem For every matrix A Mat(m, n, F), there exists a unique reduced row echelon form matrix C such that, for some invertible S, SA = C.
Uniqueness of reduced row echelon form (7) Theorem For every matrix A Mat(m, n, F), there exists a unique reduced row echelon form matrix C such that, for some invertible S, SA = C. Note that row operations, and in particular multiplication by invertible matrices, leave row space unchanged. We prove:
Uniqueness of reduced row echelon form (7) Theorem For every matrix A Mat(m, n, F), there exists a unique reduced row echelon form matrix C such that, for some invertible S, SA = C. Note that row operations, and in particular multiplication by invertible matrices, leave row space unchanged. We prove: Theorem Let U Mat(1, n, F). Then there exists a unique red. row ech. form matrix C such that rowspace(c) = U.
Uniqueness of reduced row echelon form (7) Theorem For every matrix A Mat(m, n, F), there exists a unique reduced row echelon form matrix C such that, for some invertible S, SA = C. Note that row operations, and in particular multiplication by invertible matrices, leave row space unchanged. We prove: Theorem Let U Mat(1, n, F). Then there exists a unique red. row ech. form matrix C such that rowspace(c) = U. Proof (on board): The last row of the matrix is the unique nonzero vector in row space with as many zeros as possible followed by a 1.
Uniqueness of reduced row echelon form (7) Theorem For every matrix A Mat(m, n, F), there exists a unique reduced row echelon form matrix C such that, for some invertible S, SA = C. Note that row operations, and in particular multiplication by invertible matrices, leave row space unchanged. We prove: Theorem Let U Mat(1, n, F). Then there exists a unique red. row ech. form matrix C such that rowspace(c) = U. Proof (on board): The last row of the matrix is the unique nonzero vector in row space with as many zeros as possible followed by a 1. The remaining rows span the complementary subspace, call it U, which has a 0 in the pivot entry of the last row.
Uniqueness of reduced row echelon form (7) Theorem For every matrix A Mat(m, n, F), there exists a unique reduced row echelon form matrix C such that, for some invertible S, SA = C. Note that row operations, and in particular multiplication by invertible matrices, leave row space unchanged. We prove: Theorem Let U Mat(1, n, F). Then there exists a unique red. row ech. form matrix C such that rowspace(c) = U. Proof (on board): The last row of the matrix is the unique nonzero vector in row space with as many zeros as possible followed by a 1. The remaining rows span the complementary subspace, call it U, which has a 0 in the pivot entry of the last row. By induction on dim(rowspace)(u), the remaining rows are the unique red. row ech. form matrix with U as its rowspace.
Eigenvalues (8) Definition A (nonzero) eigenvector v V of T L(V ) of eigenvalue λ is a solution of the equation Tv = λv.
Eigenvalues (8) Definition A (nonzero) eigenvector v V of T L(V ) of eigenvalue λ is a solution of the equation Tv = λv. Ambiguity: Sometimes eigenvector implies nonzero, and sometimes we allow the zero eigenvector. However:
Eigenvalues (8) Definition A (nonzero) eigenvector v V of T L(V ) of eigenvalue λ is a solution of the equation Tv = λv. Ambiguity: Sometimes eigenvector implies nonzero, and sometimes we allow the zero eigenvector. However: Definition We call λ F an eigenvalue of T if there exists a nonzero eigenvector v of T of eigenvalue λ.
Eigenvalues (8) Definition A (nonzero) eigenvector v V of T L(V ) of eigenvalue λ is a solution of the equation Tv = λv. Ambiguity: Sometimes eigenvector implies nonzero, and sometimes we allow the zero eigenvector. However: Definition We call λ F an eigenvalue of T if there exists a nonzero eigenvector v of T of eigenvalue λ. So an eigenvalue of T means there is a nonzero eigenvector.
Eigenvalues (8) Definition A (nonzero) eigenvector v V of T L(V ) of eigenvalue λ is a solution of the equation Tv = λv. Ambiguity: Sometimes eigenvector implies nonzero, and sometimes we allow the zero eigenvector. However: Definition We call λ F an eigenvalue of T if there exists a nonzero eigenvector v of T of eigenvalue λ. So an eigenvalue of T means there is a nonzero eigenvector. Definition Given any λ F, the λ-eigenspace of T is the collection of all eigenvectors of T with eigenvalue λ, together with the zero vector.
Eigenvalues (8) Definition A (nonzero) eigenvector v V of T L(V ) of eigenvalue λ is a solution of the equation Tv = λv. Ambiguity: Sometimes eigenvector implies nonzero, and sometimes we allow the zero eigenvector. However: Definition We call λ F an eigenvalue of T if there exists a nonzero eigenvector v of T of eigenvalue λ. So an eigenvalue of T means there is a nonzero eigenvector. Definition Given any λ F, the λ-eigenspace of T is the collection of all eigenvectors of T with eigenvalue λ, together with the zero vector. Then λ is an eigenvalue of T iff the λ-eigenspace is nonzero.
Eigenspaces are vector spaces (9) Proposition The λ-eigenspace of T is a vector space.
Eigenspaces are vector spaces (9) Proposition The λ-eigenspace of T is a vector space. Proof: If u and v are eigenvectors of eigenvalue λ, then T (u + v) = T (u) + T (v) = λ(u + v), so u + v is an eigenvector of eigenvalue λ. The rest is similar.
Eigenspaces are vector spaces (9) Proposition The λ-eigenspace of T is a vector space. Proof: If u and v are eigenvectors of eigenvalue λ, then T (u + v) = T (u) + T (v) = λ(u + v), so u + v is an eigenvector of eigenvalue λ. The rest is similar. Theorem (Theorem 5.10) If F = C, and V is f.d. and nonzero, then every T L(V ) has an eigenvalue.
Eigenspaces are vector spaces (9) Proposition The λ-eigenspace of T is a vector space. Proof: If u and v are eigenvectors of eigenvalue λ, then T (u + v) = T (u) + T (v) = λ(u + v), so u + v is an eigenvector of eigenvalue λ. The rest is similar. Theorem (Theorem 5.10) If F = C, and V is f.d. and nonzero, then every T L(V ) has an eigenvalue. Proof: This was in the warm-up exercise!
Real transformations (10) However, if F = R, then not all linear transformations admit an eigenvalue. Example?
Real transformations (10) However, if F = R, then not all linear transformations admit an eigenvalue. Example? We saw it already on PS1: the 90 rotation of R 2 does not!
Real transformations (10) However, if F = R, then not all linear transformations admit an eigenvalue. Example? We saw it already on PS1: the 90 rotation of R 2 does not! Theorem (To be proved later!) Suppose F = R, T L(V ), and V is f.d. and nonzero. Then there exists a subspace U V such that dim U {1, 2}, and T (U) U.
Real transformations (10) However, if F = R, then not all linear transformations admit an eigenvalue. Example? We saw it already on PS1: the 90 rotation of R 2 does not! Theorem (To be proved later!) Suppose F = R, T L(V ), and V is f.d. and nonzero. Then there exists a subspace U V such that dim U {1, 2}, and T (U) U. Example: for the rotation above, we can take U = V = R 2.
Invariant subspaces (11) This motivates: Definition Let T L(V ). An invariant subspace U V is a subspace such that T (u) U for all u U.
Invariant subspaces (11) This motivates: Definition Let T L(V ). An invariant subspace U V is a subspace such that T (u) U for all u U. Examples: V itself, {0}, eigenspaces.
Invariant subspaces (11) This motivates: Definition Let T L(V ). An invariant subspace U V is a subspace such that T (u) U for all u U. Examples: V itself, {0}, eigenspaces. Proposition The following are equivalent: (a) v is a nonzero eigenvector of T ; (b) Span(v) is a one-dim. invariant subspace.
Invariant subspaces (11) This motivates: Definition Let T L(V ). An invariant subspace U V is a subspace such that T (u) U for all u U. Examples: V itself, {0}, eigenspaces. Proposition The following are equivalent: (a) v is a nonzero eigenvector of T ; (b) Span(v) is a one-dim. invariant subspace. Proof: (a) implies (b): T (av) = aλv Span(v) for all a F;
Invariant subspaces (11) This motivates: Definition Let T L(V ). An invariant subspace U V is a subspace such that T (u) U for all u U. Examples: V itself, {0}, eigenspaces. Proposition The following are equivalent: (a) v is a nonzero eigenvector of T ; (b) Span(v) is a one-dim. invariant subspace. Proof: (a) implies (b): T (av) = aλv Span(v) for all a F; (b) implies (a): If T (v) Span(v) then T (v) = λv for some λ F. Also v is nonzero since Span(v) {0}.
Invariant subspaces (11) This motivates: Definition Let T L(V ). An invariant subspace U V is a subspace such that T (u) U for all u U. Examples: V itself, {0}, eigenspaces. Proposition The following are equivalent: (a) v is a nonzero eigenvector of T ; (b) Span(v) is a one-dim. invariant subspace. Proof: (a) implies (b): T (av) = aλv Span(v) for all a F; (b) implies (a): If T (v) Span(v) then T (v) = λv for some λ F. Also v is nonzero since Span(v) {0}. Now the theorem on the last slide says that if F = R then T L(V ) admits a nonzero invariant subspace of dimension 2.
Invariant subspaces (11) This motivates: Definition Let T L(V ). An invariant subspace U V is a subspace such that T (u) U for all u U. Examples: V itself, {0}, eigenspaces. Proposition The following are equivalent: (a) v is a nonzero eigenvector of T ; (b) Span(v) is a one-dim. invariant subspace. Proof: (a) implies (b): T (av) = aλv Span(v) for all a F; (b) implies (a): If T (v) Span(v) then T (v) = λv for some λ F. Also v is nonzero since Span(v) {0}. Now the theorem on the last slide says that if F = R then T L(V ) admits a nonzero invariant subspace of dimension 2. Corollary (preview): If dim V is odd (F = R), then T has an eigenvalue.