Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column form): x + x =. The matrix form: Ax = b [ ] [ ] [ ] 4 4 x with A =, x =, and b =. 4 4 x. Matrix operations: addition/subtraction, scalar multiplication, matrix product, (AB = BA?) transpose, and inversion. The identity matrix, diagonal matrix, upper triangular matrix, and lower triangular matrix. (AB) T = B T A T. (AB) = B A (why?) Is the inverse of a diagonal (upper or lower triangular) matrix still diagonal (upper or lower triangular)?. Elementary row reductions and the corresponding elementary matrices. (Their relations?) Pivots. Echelon form U. Reduced echelon form R. Permutation matrix. Gaussian elimination. Use elementary row reduction to solve Ax = b and to find the inverse of an invertible matrix. 4 8 Exercise. Reduce A = 4 6 8 to an echelon form and the reduced echelon 6 9 0 form, and find out the pivots. Find the inverse of B = 0. 4 8 4. Matrix factorization: A = LU, A = LDU, P A = LU, and P A = LDU. The symmetric factorization A = LDL T for a symmetric matrix A. Exercise. Use A = LU to solve Ax = b. LU and LDU-factorize A =. Use x the LU factorization to solve 0 5 7 y =. 6 9 8 z 5 5. Operation count (page 4), Matlab (page 9), and Section.7 will not be covered in the exam. Chapter. Vector Spaces. The concept of a vector space. Examples: R n, matrix space, polynomial space, space of continuous functions, space of sequences of real numbers, etc. Subspaces. Check if a given subset of a vector space is a subspace.
Exercise. Is the set of polynomials of degree a subspace of the space of all polynomials? Is the set of all matrices with determinant equal to 0 a subspace of the space of all matrices? Give a matrix A, why the set of all solutions x to Ax = 0 is a subspace? What is the zero vector of the vector space that consists of all matrices?. Linear combination. Linear dependence and independence. Span. Vectors u,..., u k are linearly dependent if and only if one of them is a linear combination of the others. Why? Exercise. If u, u, u are linearly independent, and au + u = u + bu + cu (a, b, c are numbers), then a =? b =? and c =? If each of u,..., u 5 is a linear combination of v,..., v 4 and each of v,..., v 4 is a linear combination of w,..., w, then each of u,..., u 5 is a linear combination of w,..., w, correct? Why n + vectorsinr n are always linearly dependent? Is a linear combination of 0, 5, and 7? Are 5 6 9 8 6,, and 9 7 4 linearly independent?. Solve Ax = 0 : free variables (if any). Solve Ax = b. Relation between solutions to Ax = b and to Ax = 0. How to check if Ax = b is consistent (i.e., there is at least one solution)? Rank of A = the number of pivots. Rank (A) = Rank (A T ). [ ] [ ] 0 5 Exercise. For A = and b =, solve Ax = 0 and Ax = b. Find conditions on b, b, b so that the system of equations 5 4 y = b 0 4 x b is solvable; 4 9 8 z b and solve the system of equations when the conditions are satisfied. Rank (AB) = Rank (A) Rank (B)? Rank (A + B) = Rank (A + B)? 4. Basis: A group of vectors that are linearly independent and that span the entire space (i.e., any vector is a linear combination of these vectors. If u,..., u m and v,..., v n are two bases for a vector space, then m = n. Why? Dimension = number of a vectors in a basis. Exercise. Are 0 and linearly independent? If so, find another vector in R 0 so that all the three vectors together are still linearly independent. How to remove vectors, as fewer as possible, from a list of 5 vectors in R so that the remaining vectors are linearly independent? What is the dimension of the vector of polynomials of degree 4? Find a basis for this polynomial space. Final a basis for the vector space of all 4 matrices. What is the dimension of the space of all 5 5, symmetric matrices? 5. The four fundamental subspaces of a given matrix A : the column space of A, the null space of A, the row space of A, and the left null space of A. How to find their
dimensions and bases? Dimension of C(A) + dimension of N(A) = number of columns. Ax = b means b is in C(A), correct? Rank-one matrices. Equivalent statements for an invertible matrix. Exercise. Let A = 6 9 7. Find a basis for C(A). Find a basis for N(A). 4 0 0 Describe the four spaces of the matrix A = 0 0. If B = 4 a 0 0 0 8 4 4 rank-one matrix? If so, find vectors a and b such that B = ab T. 6. Linear transformations. How to check a transformation is linear? If T : V W is a linear transformation, v,..., v n and w,..., w m are bases for V and W, respectively. How to find a the matrix A that represents T? Is A an m n or n m matrix? Exercise. Find the matrix for the reflection about the x-axis. Find the matrix that rotates any vector by π/4 counterclockwise. Let P denotes the space of all polynomials of degrees. Find the matrix that represents the linear transformation defined by differentiation with respect to the basis, x, and x. 7. Section.5 will not be covered in the exam. Chapter. Orthogonality. Inner product. Orthogonal or perpendicular vectors. Length of vector. Unit vectors. Distance. Nonzero, mutually orthogonal vectors are linearly independent? Why? Definition of the angle θ between two nonzero vectors in R n. Law of cosines. Why cos θ? The Cauchy Schwarz inequality. Exercise. Given a = and b =. Find a, a b, the cosine of the angle between a and b. Determine if a and b are orthogonal. If u and v are orthogonal, the u v = u + v. How to prove this? Show that for any two vectors x and y that x + y x + y.. Orthogonal subspaces. The orthogonal complement of a subspace. If two subspaces V and W are orthogoal, does it mean that W is the orthogonal complement of V? Why? In R n, if W = V then V = W and dim V + dim W = n. Given an m n matrix A. N(A) = C(A T ) and N(A T ) = C(A) : correct? why? Exercise. Find the orthogonal complement of the subspace spanned by a = 0 and b =.
. Projection onto a line (or a vector) and the corresponding projection matrix P. P T = P and P = P. Rank (P ) =? Projection onto a subspace, particular the column space of a matrix A. What is the projection matrix now? Exercise. Given a = and b =. Find the projection of b onto a. What is the matrix of the projection onto a? 4. Least-squares problem: formulation as a minimization problem, normal equation, the matrix A T A, relation to projection onto a subspace, geometrical interpretation. 0 Exercise. Solve Ax = b by least squares, and find p = Aˆx, if A = 0 and b =. For this A, find the projection matrix for the orthogonal projection onto the 0 column space of A. 5. Orthonormal vectors. Orthogonal matrices and their properties: Q T Q = I and Qx = x. Gram Schmidt orthogonalization process. A = QR factorization. 0 0 Exercise. Apply the Gram Schmidt process to a = 0, b =, and c =, and write the result in the form A = QR. QR factorize the matrix 0 0. Why 0 0 the determinant of an orthogonal matrix is always or? 6. The following will not be covered in the exam: Weighted least-squares (in Section.); Rectangular Matrices with Orthonormal Columns (in Section.4); Function Spaces and Fourier Series (in Section.4); and Section.5. Chapter 4. Determinants. What are the three basic properties that define the determinant of an n n matrix? What is the determinant of a diagonal matrix? an upper or lower triangular matrix? a matrix with one zero-row? a matrix with two identical rows? det(a T ) = det A? det(ab) = det A det B? det(a + B) = det A + det B? det(ca) = c det A (c is a number)? If rows (or columns) of A are linearly dependent, then det A = 0; correct?. Compute determinants using various kinds of properties of determinants and formulas: by elementary row reductions (be careful: such a reduction can change the value of the determinant); by expansion along one row or one column (the co-factor expansion); and by some formulas (especially for n = or. 4
Exercise. Compute the determinants of 0 0 4 0 0,, 0 4 5 4 and 0 0 0. 0. Cramer s rule for solving system of linear equations. Why this works? { x + x =, Exercise. Use Cramer s rule to solve the system of equations: x + 4x = 4. 4. Compute areas and volumes using determinants. Determine if A is invertible by det A. Exercise. Find the volume of the parallelogram defined by the vectors (, 0, ), (,, ), and (,, ). Exercise. Let A be the matrix with row vectors (, 0, ), (,, ), and (,, ). Calculate det A and determine if A is invertible. Chapter 5. Eigenvalues and Eigenvectors. Definition. How to compute eigenvalues and eigenvectors? Eigenvalues of a diagonal matrix and an upper (or lower) triangle matrix. If det A = 0 then 0 is an eigenvalue of A: correct? If λ is an eigenvalue of A then λ is an eigenvalue of A ; correct? Eigenvectors corresponding to different eigenvalues of a matrix are always linearly independent; correct? Why?. If A is an n n matrix, then its characteristic polynomial p(λ) = det(a λi) is a polynomial of degree exactly n. What is the coefficient of λ n and that of λ n of p(λ)? What is the constant term of p(λ)? The sum of diagonal entries of A equals the sum of all eigenvalues of A, right? Why? The det A equals the product of all eigenvalues of A, right? Why?. Diagonalization of a matrix: definition and calculation. An n n matrix is diagonalizable means the existence of n linearly independent eigenvectors of this matrix, right? why? How to compute A k if A is diagonalizable? What is the definition of e A? How to compute e A if A is diagonalizable? Exercise. Find a matrix that is not diagonalizable. [ ] Exercise. Diagonalize the following matrices A: and. Then compute A, A 0, and e A 4. Exercise. If A is Diagonalizable and all the eigenvalues of A have the absolute values strictly less than, then lim k A k = 0 (the zero matrix). Prove it. Exercise. Let H 0 = 0, H =, and H n+ = (H n+ + H n )/ (n = 0,,... ). Find a general expression of H n and find the limit lim n H n. 5
4. Complex matrices. Inner product and length of vectors. Hermitian. If A is Hermitian, then for any vector (complex-valued) x: x H Ax is always real. Why? [ ] [ ] i i Exercise. Let A = and x =. Find A + i H. Is A Hermitan? Find x H Ax. 5. What is an orthogonal matrix? What is a unitary matrix? If A is orthogonal or unitary, then Ax = x ; correct? If λ is an eigenvalue of an orthogonal or unitary matrix, then λ = ; why? What is A if A is orthogonal or unitary? 6. Properties of eigenvalues of a realy, symmetric or Hermitian matrix: always real, and eigenvectors corresponding to different eigenvalues are orthogonal. Spectral Theorem (page 97): A = QΛQ T for a real and symmetric A with Q orthogonal and Λ diagonal; A = UΛU H for a Hermitan A with U unitary and Λ diagonal. The table Real versus Complex on page 88. [ ] 4 Exercise. Find orthogonal Q and diagonal Λ for A = [ ] [ 0 i 0 + i diagonal Λ for A = or A = + i + i i ] 4.. Find unitary Q and 7. Similar matrices: B = M AM is similar to A. Diagonalizable A is similar to a diagonal matrix. Similar matrices have same eigenvalues and the same characteristic polynomial. Diagonalization of Hermitan matrices: The Spectral Theorem. Triangularization of any matrix (Schur s lemma): For any A, there is unitary U such that U AU = T, a triangular matrix. Jordanization of any matrix: A is similar to its Jordan form. What is a Jordan form? Exercise. If a 5 5 matrix A has the eigenvalues λ = λ = 0 with one linearly independent eigenvector, and λ = λ 4 = λ 5 = with only two linearly independent eigenvectors. Find all Jordan blocks of A and all possible Jordan forms of A. 8. The following will not be covered in the exam: Examples of differential equations in Section 5. (pages, 4, and 7); Examples,, and 4 on pages 60 6; differential equations in Section 5.4; stabilities in Section 5. and Section 5.4, and Example 6 on page 00. Chapter 6. Positive Definite Matrices. Associate a symmetric matrix with a quadratic form. Definition of positive definiteness and semi-definiteness of a symmetric matrix. Tests for the positive definiteness of a real symmetric matrix A: () By the definition: x T Ax > 0 for any x 0; (This usually involves completing the squares.) () All eigenvalues are positive; () All principal submatrices have positive determinants; and (4) A = R T R for nonsingular matrix R. 0 Exercise. Let A = 0 4. Write down the quadratic form x T Ax. 4 6
Exercise. Decide if each of the following matrices is positive definite: 9 0 0 0 0,, 0. 0 5 Exercise. If A is real, symmetric and positive definite, then each of its diagonal entries must be positive.. What are the singular values of a matrix? Singular value decomposition (SVD): A = UΣV T. How to find Σ, V, and U? Exercise. If A is a matrix and the eigenvalues of A T A are 8, 4, and 0, what is Σ? If A is a 4 matrix and the eigenvalues of A T A are 8, 4,, and 0, what is Σ?. How to find the minimum of P (x) = (/)x T Ax b T x? How to find minimum of P (x) = (/)x T Ax b T x under the constraint Cx = d? 4. What is the Rayleigh quotient for a symmetric matrix A? Exercise. Find the minimum value of R(x) = x x x + x. x + x 5. The following will not be covered in the exam: pages 6 of Section 6.; Applications of SVD in Section 6.; Intertwining of the eigenvalues in Section 6.4; and Section 6.5. 7