LINEAR ALGEBRA REVIEW SPENCER BECKER-KAHN Basic Definitions Domain and Codomain. Let f : X Y be any function. This notation means that X is the domain of f and Y is the codomain of f. This means that for every x X, there is exactly one element f(x) Y. Range. The range of f is defined to be the set range(f) : = {y Y There exists x X with f(x) = y} = {f(x) Y x X}. This is a subset of the codomain of f. One-to-One. We say that f is one-to-one if f(x ) = f(x 2 ) = x = x 2. This means that for every y Y, there is at most one x X with f(x) = y. Onto. We say that f is onto if for every y Y, there is x X with f(x) = y. This means that the range of f is equal to the codomain of f. Subspaces. A subset S R n is said to be a subspace of R n if () u, v S = u + v S; and (2) u S = λ u S for all λ R. e.g. A line through the origin in R 2. Or a line or a plane through the origin in R 3. All subspaces look like this in some sense: They are flat and infinite and always go through the origin. Linear Combination. Let { u,..., u k } be a set of vectors in R n. A linear combination of the vectors u,... u k is any vector of the form where a,..., a k are real numbers. a u + + a k u k, Span. Let { u,..., u k } be a set of vectors in R n. The span of { u,..., u k } is defined to be the set of all linear combinations of u,... u k and is denoted span u,..., u k. Therefore span u,..., u k = { a u + + a k u k R n a,..., a k R } Linear Independence. Let { u,..., u k } be a set of vectors in R n. { u,..., u k } is said to be linearly independent if a u + + a k u k = = a = a 2 = = a k =. The set i.e. One might think that the only way to make a linear combination of the vectors equal to zero is to take all the scalars equal to zero.
2 SPENCER BECKER-KAHN Basis. Let S be a subspace of R n. A set B = { u,..., u k } of vectors in R n is said to be a basis of S if () B is linearly independent; and (2) span B = S. It can be shown that for a fixed subspace S, the number of elements in any basis is always the same. Dimension. Given a subspace S of R n, the dimension of S is the number of elements in any basis of S. Linear Transformation. A function T : R k R n is said to be a linear transformation (or simply linear ) if () T ( u + v) = T ( u) + T ( v) for all u, v R k ; and (2) T (λ u) = λt ( u) for all u R k and λ R. The range of a linear transformation is a subspace of the codomain. Kernel. The kernel of T is defined to be the set ker(t ) := { x R k T ( x) = } The kernel of a linear transformation is a subspace of the domain. Matrix Algebra. Let A be an (n k) matrix with columns a,..., a k R n. For x = (x,..., x k ) R k we define A x = x a + + x k a k. If B is a (k m) matrix with columns b,..., b m R k, then we can form the matrix product AB which is an (n m) matrix with columns A b,..., A b m. Remember that matrix multiplication is not commutative, which means that for two (n n) matrices A and B, in general AB is not equal to BA. The Identity Matrix. The identity matrix I n is defined to be the (n n) matrix whose entry in the i th row and j th column is equal to if i = j and equal to zero otherwise. Inverse Matrix. The (n n) square matrix A is said to be invertible if there exists a square matrix B such that AB = BA = I n. Given an invertible matrix A, there is exactly one matrix B that satisfies the above equations. We call this matrix the inverse of A and write A = B. Transpose Matrix; Symmetric Matrices. Given an (n k) matrix A, the transpose of A, denoted A T, is the (k n) matrix that is obtained from A by interchanging the rows and columns of A. A (square) matrix A is said to be symmetric if A = A T.
LINEAR ALGEBRA REVIEW 3 Null Space and Nullity. Let A be an (n k) matrix with columns a,..., a k R n. The null space of A is defined to be null(a) := { x R k A x = }. The nullity of A is the dimension of the null space of A, i.e. nullity(a) := dim null(a). Column Space and Rank. Let A be an (n k) matrix with columns a,..., a k R n. The column space of A is defined to be the span of the columns of A, i.e. it is defined to be the set col(a) := span a,..., a k. The rank of A is the dimension of the column space of A, i.e. rank(a) := dim col(a). Row Space. The row space of A is the span of the rows of A and is denoted row(a). Theorem. (Row Rank Equals Column Rank) For any matrix A, the dimension of the row space of A is equal to the dimension of the column space of A, i.e. both numbers are equal to the rank of A. Theorem. (Rank-Nullity Theorem) Let A be an (n k) matrix. We have that rank(a) + nullity(a) = k. If T : R k R n is a linear transformation, then dim range(t ) + dim ker(t ) = k. The Determinant of a 2 2 Matrix. For a 2 2 matrix [ ] a b B := c d we define the real number det B := ad bc. The Determinant of an n n Matrix. For a general n n matrix A, the determinant is defined inductively. We let M ij denote the (n ) (n ) matrix obtained by deleting the i th row and j th column of A (i.e. by deleting the row and column containing a ij ). This is called the minor of a ij. Then we define the cofactor of a ij to be Now we define det A := C ij := ( ) i+j det M ij. n a j C j = a C + a 2 C 2 + + a n C n. j= This means the definition of det A relies on the determinant of smaller matrices having already been defined. This is OK because we can explicitly define the determinant of a 2 2 matrix.
4 SPENCER BECKER-KAHN Theorem. (Expanding Determinant About Any Row or Column.) Let A be an n n matrix. For any i =,... n we have n det A = a ij C ij = a i C i + a i2 C i2 + + a in C in. j= And for any j =,... n we have n det A = a ij C ij = a j C j + a 2j C 2j + + a nj C nj. i= Theorem (Multiplicative Property of Determinant). Let A and B be any two n n matrices. Then det(ab) = det(a) det(b). Theorem. Let A be an n n matrix. Then A is invertible det(a). Remarks. The following table is a small dictionary between matrices and linear transformations: Let A be an (n k) matrix with columns a,..., a k R n and let T : R k R n be a linear transformation given by T ( x) = A x. Also let e,..., e k denote the standard basis vectors in R k, i.e. e =, e 2 =,..., e k =.. Then the thing on the left is the same as the thing on the right: Matrices Linear Transformations A x = c always has at most one solution T is one-to-one A x = c always has at least one solution T is onto a j T ( e j ) col(a) range(t ) rank(a) dim range(t ) null(a) ker(t ) nullity(a) dim ker(t ) Echelon Form One of the main things that one learns in this class is how to put matrices into echelon form and what that means. The nature of echelon form is such that if B is a matrix in echelon form, then: The non-zero rows of B are a basis for row(b). The pivot columns of B are a basis for col(b)..
LINEAR ALGEBRA REVIEW 5 Elementary Row Operations. Let A be an (n k) matrix with columns a,..., a k R n and rows r,..., r n R k. There are three types of elementary row operation that one can perform on the matrix A: () For some i {,..., n} and some c R: Replace r i by c r i. (2) For some i, j {,..., n}: Swap r i r j. (3) For some i, j {,..., n} and some c R: Replace r i by r i + c r j. Equivalent Matrices. If the matrix B is obtained by starting with A and performing some finite number of elementary row operations then we say that A and B are equivalent and write A B. Elementary Matrices. A matrix E that is obtained by performing one elementary row operation on the n n identity matrix I n is called an elementary matrix. The following facts can now all easily be seen to be true: () Each elementary matrix is invertible. (2) Each elementary row operation performed on A can be realized by first performing the operation on the n n identity matrix I n to produce an elementary matrix E and then computing the matrix product EA. (3) If A B, then there is a finite sequence E,..., E M of elementary matrices for which B = E M E A. (4) If A B, then there is an invertible matrix R for which (5) If A B then row(a) = row(b). B = RA. Solving Linear Systems. Fix a vector c R n and suppose that you are trying to solve the equation A x = c. Let B denote the reduced echelon form of the matrix A, let b,..., b k denote the columns of b and let R denote the invertible matrix for which B = RA. Now, by multiplying on the left by the matrix R, it is clear that From this we get the following facts: A x = c B x = R c. () The solution set to [ A c ] is the same as the solution set to [ B R c ]. (2) For any set of scalars x,..., x k, we have x a + + x k a k = x b + + x k bk =. (3) The set { a j bj is a pivot column of B} is a basis for col(a) (i.e. for each j =,..., k, the column a j belongs to this set if and only if b j is a pivot column).
6 SPENCER BECKER-KAHN Spectral Theory Eigenvectors and Eigenvalues. Let A be an n n matrix. The number λ R is said to be an eigenvalue of A if there exists a non-zero vector v such that A v = λ v. Any such vector v satisfying this equation is called an eigenvector of A. The set E λ (A) := { x R n A x = λ x} is the λ-eigenspace of A. (So every non-zero elements of E λ (A) is a λ-eigenvector of A.) Diagonalization. A square matrix A is said to be diagonalizable if there exists an invertible matrix P such that P AP is a diagonal matrix. We say that P diagonalizes A. Theorem. An n n matrix is diagonalizable if and only if there exists a basis of R n that consists entirely of eigenvectors of A. And if A is diagonalized by P, then the columns of P must be a basis of eigenvectors of A. Warning: D = P AP A = P DP. So if the only thing you remember is columns are the eigenvectors, then you risk making a mistake about whether or not you are referring to P or P. (One way to check is that since the standard basis vectors are eigenvectors of a diagonal matrix: P AP e i = λ e i, which means that A(P e i ) = λ(p e i ), which means that the columns of P are eigenvectors of A). The Characteristic Polynomial. Given an n n matrix A, the function is a polynomial in λ of degree n. Then: λ det(a λi n ) Theorem. The number λ R is an eigenvalue of A if and only if det(a λi n ) =.