ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2 Given any m n matrix A, the row rank of A is always equal to the column rank of A. Fact 2.1 Given a set of vectors x 1 x m, if one of the vectors, say x k, is the zero vector, then the given vectors are linearly dependent. Fact 2.2 A set of vectors is linearly dependent if and only if one of the vectors in the set can be written as a linear combination of the others. Fact 2.3 Any vector which can be written as a linear combination of linearly independent vectors has a unique set of expansion coefficients. Fact 2.4 Given x 1 x k, let S be the set of all possible linear combinations of x 1 x k. Then S is a subspace of R n, and S is said to be the subspace generated by x 1 x k. Fact 2.5 The set of vectors e 1,, e n introduced Example 2.1 is a basis for R n. This set of vectors is called the standard basis for R n. Fact 2.6 Let S be a subspace with a given basis b 1 b m. Then every basis for S must contain exactly m vectors.
Fact 2.7 Let S be an m-dimensional subspace of R n. Then no set of linearly independent vectors in S has more than m elements. Furthermore, any set of m linearly independent vectors in S is a basis for S. Fact 2. 8 Let S be a m-dimensional subspace of R n and let b 1, b k be linearly independent vectors in S for some k m. Then additional vectors b k+1, b k+2,, b m can be found such that B = {b 1,, b m } is a basis for S. Fact 2.9 Let S be an m-dimensional subspace of R n and let B = {b 1,, b p }, p m be a set of vectors which spans S. Then m vectors can be chosen from B to be a basis for S. Fact 2. 10 The set S of all linear combinations of m linearly independent vectors b 1,, b m is an m-dimensional subspace of R n. Fact 2.11 1. A + B = B + A. 2. (A + B) + C = A + (B + C). Fact 2.12 The product of a matrix A and a column vector x can be written as a linear combination of the columns of A in which the expansion coefficients are the elements of the vector x, i.e. x 1 Ax = [ a 1 a n ]. = a 1 x 1 + a 2 x 2 + + a n x n. x n
Fact 2.13 The product of a row vector y T and a matrix A can be written as a a linear combination of the rows of A in which the expansion coefficients are the elements of the vector y, i.e. y T A = [ y 1 y m ] a T 1. a T m = y 1 a T 1 + y 2 a T 2 + + y m a T m. Fact 2.14 If the matrix C = AB, then the (i, j)-element of C is the inner product of the ith row of A with the jth column of B, i.e. c ij = a T i b j. Fact 2.15 If the matrix C = AB, then C can be written as the sum of outer products of columns of A with rows of B, i.e. C = a 1 b T 1 + a 2 b T 2 + + a m b T m. (1) Fact 2.16 The matrix product AB can be written column-wise as A times the columns of B AB = A [ b 1 b n ] = [ Ab 1 Ab 2 Ab n ]. Fact 2.17 The matrix product AB can be written row-wise as rows of A times B AB = a T 1. a T p B = a T 1 B.. a T p B
Fact 2.18 Let the matrices A and B be partitioned as follows A 11 A 1l A =... A k1 A kl, B = B 11 B 1n... B m1 B mn where A ij is a submatrix of A with dimensions α i δ j and B ij is a submatrix of B with dimensions β i γ j. Then the addition, multiplication, and transpose operations can be performed as follows: 1. If k = m, α i = β i, and δ j = γ j, then C 11 C 1l A + B =... C k1 C kl where C ij = A ij + B ij. 2. If l = m and δ i = β i then C 11 C 1n AB =... C k1 C kn where l C ij = A iq B qj. q=1 3. A T = A T 11 A T k1.... A T 1l A T kl Fact 2.19 Let A and B be nonsingular matrices having the same dimensions. Then (AB) 1 = B 1 A 1.
Fact 2.20 If A is a nonsingular matrix then (A 1 ) T = (A T ) 1 def = A T. Fact 2. 21 A set of vectors x 1,, x k R n is linearly independent if and only if the determinant of the Gram matrix, G, associated with these vectors is not equal to zero. G = X T X where X is a matrix whose columns are the given vectors. Fact 2.22 Let R be an m-dimensional subspace of R n and let x 1, x 2,, x m be a basis for R. Let y be a vector in R n and define W to be the following matrix W def = [ x 1 x 2 x m y ]. If the columns of W are linearly independent then y R. If the columns of W are linearly dependent the y R. Fact 2.23 The columns (or rows) of a n n matrix A are linearly independent if and only if det(a) 0. Fact 2.24 x = 0 x = 0. Fact 2.25 An orthogonal (or orthonormal) set of nonzero vectors is linearly independent. Fact 2.26 null(a) = row(a). Fact 2.27 The rank of an m n matrix A is less than or equal to min(m, n).
Fact 2.28 Given an m n matrix A, rank(a) + nullity(a) = n. Fact 2.29 If λ is an eigenvalue of A then αλ is an eigenvalue of αa. Fact 2.30 If λ is an eigenvalue of A then α + λ is an eigenvalue of αi + A. Fact 2.31 Similar matrices (e.g. A and TAT 1 ) have the same eigenvalues. Fact 2.32 The scalar λ is an eigenvalue of A if and only if det(λi A) = 0. Fact 2.33 Let A be an n n matrix. Then det(λi A) is a monic polynomial in λ of degree n. It is denoted a(λ) and called the characteristic polynomial of A. We have a(λ) def = det(λi A) = λ n + a 1 λ n 1 + + a n. Fact 2.34 An n n matrix A has n eigenvalues which are the roots of the characteristic polynomial a(λ). Fact 2.35 Eigenvectors corresponding to distinct eigenvalues are linearly independent. Fact 2.36 The determinant of a matrix equals the product of its eigenvalues. Fact 2.37 A matrix is nonsingular if and only if all its eigenvalues are nonzero.
Fact 2.38 A matrix and its transpose have the same eigenvalues. Fact 2.39 The eigenvalues of a triangular matrix are equal to the diagonal elements of the matrix. Fact 2.40 The eigenvalues of a block-triangular matrix A are the union of the eigenvalues of the matrices on the main diagonal of A. Fact 2.41 1. If ρ(w) ρ(a) then no solutions exist. 2. If ρ(w) = ρ(a) then at least one solution exists. (a) If ρ(w) = ρ(a) = n, then there is a unique solution for x. (b) If ρ(w) = ρ(a) = r < n, then there is an infinite set of solution vectors parameterized by n r free variables. 3. If ρ(a) = m, a solution exists for every possible right hand side vector y. This case can only occur if m n. From 2., the solution is unique if m = n and non-unique if m < n.