Lecture 01 Introduction & Matrix-Vector Multiplication Songting Luo Department of Mathematics Iowa State University MATH 562 Numerical Analysis II Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 1 / 14
Outline 1 Course Information 2 Matrix-Vector Multiplication Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 2 / 14
Course Description What is numerical analysis? numerical linear algebra? The study of algorithms for the problems of continuous mathematics. Topics: Solutions of linear equations. Matrix factorization and decomposition. Conditioning, stability, and efficiency. Computation of eigenvalues and eigenvectors. Solution of non-linear equations. Prerequisite/Co-requisite: Calculus, ODE, PDE, MATH 317 (Linear Algebra). Basic programming tools such as Matlab ( or GNU Octave). Required Textbook: Numerical Linear Algebra, by Lloyd N. Trefethen and David Bau, III, SIAM, 1997, ISBN 0-89871-361-7. Classpage (Syllabus): http://orion.math.iastate.edu/luos/teaching /MATH562 16SS/MATH562 16SS.html Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 3 / 14
Definition Matrix-vector product b Ax: b i nÿ a ij x j All entries belong to C, the field of complex numbers. The space of m-vectors is C m, and the space of m ˆ n matrices is C mˆn. The map x Ñ Ax is linear, which means that for any x, y P C n, and any α P C: Apx ` yq Ax ` Ay, j 1 Apαxq αax. Conversely, every linear map can be expressed as multiplication by a matrix. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 4 / 14
Linear Combination Alternatively, matrix-vector product can be viewed as b Ax nÿ x j a j j 1 i.e., b is a linear combination of column vectors of A. Two different views of matrix-vector products: scalar operations: A acts on x to produce b: b i ř n j 1 a ijx j vector operations: x acts on A to produce b: b Ax ř n j 1 x ja j If A P C mˆn, Ax can be viewed as a linear mapping from C n to C m. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 5 / 14
Matrix-Matrix Multiplication If A is l ˆ m and C is m ˆ n, then b AC is l ˆ n, with entries defined by mÿ b ij a ik c kj Written in columns, we have k 1 b j Ac j mÿ c kj a k k 1 In other word, each column of B is a linear combination of the columns of A. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 6 / 14
Perspective: Vector Space Understanding matrix operations in terms of vector spaces Vector space spanned by a set of vectors is composed of linear combinations of these vectors It is closed under addition and scalar multiplication 0 is always a member of a subspace Space spanned by m-vectors is subspace of C m If S 1 and S 2 are two subspaces, then S 1 Ş S2 is a subspace, so is S 1 ` S 2, the space of sum of vectors from S 1 and S 2. (Note: is S 1 ` S 2 equivalent to S 1 Ť S2? ) Two subspaces S 1 and S 2 of C m are complementary subspaces of each other if S 1 ` S 2 C m and S 1 Ş S2 t0u. In other words, dimps 1 q ` dimps 2 q m and S 1 Ş S2 t0u Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 7 / 14
Range and Null Space Definition The range of a matrix A, written as rangepaq, is the set of vectors that can be expressed as Ax for some x. Theorem rangepaq is the space spanned by the columns of A. (Therefore, the range of A is also called the column space of A.) Definition The null space of A P C mˆn, written as nullpaq, is the set of vectors x that satisfy Ax 0. (Entries of x P nullpaq give coefficients of ř x i a i 0) Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 8 / 14
Rank Definition The column rank of a matrix is the dimension of its column space. The row rank is the dimension of the space spanned by its rows. Question: Can the column rank and the row rank be different? Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 9 / 14
Rank Definition The column rank of a matrix is the dimension of its column space. The row rank is the dimension of the space spanned by its rows. Question: Can the column rank and the row rank be different? Answer: No! We therefore simply say the rank of a matrix. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 9 / 14
Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 9 / 14 Rank D Definition The column rank of a matrix is the dimension of its column space. The row rank is the dimension of the space spanned by its rows. Question: Can the column rank and the row rank be different? Answer: No! We therefore simply say the rank of a matrix. Question: Given A P C mˆn, what is dimpnullpaqq ` rankpaq equal to?
Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 9 / 14 Rank D Definition The column rank of a matrix is the dimension of its column space. The row rank is the dimension of the space spanned by its rows. Question: Can the column rank and the row rank be different? Answer: No! We therefore simply say the rank of a matrix. Question: Given A P C mˆn, what is dimpnullpaqq ` rankpaq equal to? Answer: n (Rank-nullity theorem)
Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 10 / 14 Transpose and Adjoint D Transpose of A, denoted by A T, is the matrix B with b ij a ji Adjoint or Hermitian conjugate, denoted by A or A H, is the matrix B with b ij ā ji Note that: pabq T B T A T and pabq B A A matrix A is symmetric if A A T. It is Hermitian if A A. A matrix A is skew-symmetric if A A T. It is skew-hermitian if A A. Diagonal matrix, Upper (Lower) triangular matrix, etc..
Definition D A matrix has full rank if it has the maximal possible rank, i.e., mintm, nu Theorem A matrix A P C mˆn with m ě n has full rank if and only if it maps no two distinct vectors to the same vector. Proof. pñq Column vectors of A forms a basis of rangepaq, so every b P rangepaq has a unique linear expansion in terms of the columns of A. pðq If A does not have full rank, then its column vectors are linear dependent, so its vectors do not have a unique linear combination. Definition A nonsingular or invertible matrix is a square matrix of full rank. Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 11 / 14
Inverse Definition D Given a nonsingular matrix A, its inverse is written as A 1, and AA 1 A 1 A I Note that pabq 1 pa 1 q pa q 1, and we use A as a shorthand for it B 1 A 1 Theorem The following conditions are equivalent: (a) A has an inverse A 1 (b) rankpaq is m (c) rangepaq is C m (d) nullpaq is t0u (e) 0 is not an eigenvalue of A (f) 0 is not a singular value of A (g) detpaq 0 Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 12 / 14
Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 13 / 14 Matrix Inverse Times a Vector D When writing x A 1 b, it means x is the solution of Ax b In other words, A 1 b is the vector of coefficients of the expansion of b in the basis of columns of f A. Multiplying b by A 1 is a change of basis operations to ta 1, a 2,..., a m u from te 1, e 2,..., e m u Multiplying A 1 b by A is a change of basis operations to te 1, e 2,..., e m u from ta 1, a 2,..., a m u.
Songting Luo ( Department of Mathematics Iowa State University[0.5in] MATH562 MATH 562 Numerical Analys 14 / 14 Rank-1 Matrices Full-rank matrices are important. D Another interesting special case is rank-1 matrices. A matrix A is rank-1 if it can be written as A uv, where u and v are nonzero vectors uv is called the outer product of the two vectors, as opposed to the inner product u v