G1110 & 852G1 Numerical Linear Algebra

Size: px
Start display at page:

Download "G1110 & 852G1 Numerical Linear Algebra"

Transcription

1 The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the Householder matrix H(w.

2 Lecture notes and course material by Holger Wendland, David Kay, and others, who taught the course Numerical Linear Algebra at the University of Sussex, served as a starting point for the current lecture notes. The current lecture notes are about twice as many pages as the previous version. Apart from corrections and improvements, many new examples and some linear algebra revision sections have been added compared to the previous lecture notes.

3 Contents Introduction and Motivation iii. Motivation: An Interpolation Problem iii. Motivation: A Boundary Value Problem v Revision: Some Linear Algebra. Vectors in R n and C n Matrices Determinants of Square Matrices Inverse Matrix of a Square Matrix Eigenvalues and Eigenvectors of a Square Matrix Other Notation: The Landau Symbol Matrix Theory 5. The Eigensystem of a Square Matrix Upper Triangular Matrices and Back Substitution Schur Factorization: A Triangular Canonical Form Vector Norms Matrix Norms Spectral Radius of a Matrix Floating Point Arithmetic and Stability Condition Numbers of Matrices Floating Point Arithmetic Conditioning Stability An Example of a Backward Stable Algorithm: Back Substitution i

4 ii Contents 4 Direct Methods for Linear Systems 7 4. Standard Gaussian Elimination The LU Factorization Pivoting Cholesky Factorisation QR Factorization Iterative Methods for Linear Systems Introduction Fixed Point Iterations The Jacobi and Gauss-Seidel Iterations Relaxation The Conjugate Gradient Method 5 6. The Generic Minimization Algorithm Minimization with A-Conjugate Search Directions Convergence of the Conjugate Gradient Method Calculation of Eigenvalues 5 7. Basic Localisation Techniques The Power Method Inverse Iteration The Jacobi Method Householder Reduction to Hessenberg Form QR Algorithm

5 Introduction and Motivation The topics of this course center around the numerical solution of linear systems and the computation of eigenvalues. Let A R m n be an m n matrix and let b R m be a vector. How do we find the (approximate solution x R n to the linear system Ax = b if the matrix A is very large, say, if A is a 6 6 matrix? Unlike in linear algebra, where we have learnt under what assumptions on A and b a (unique solution exists, here the focus is on how this system should be solved with the help of a computer. In devising algorithms for the numerical solution of such linear systems, we will exploit the properties of the matrix A. If the matrix A R n n is a square matrix, then we may want to find the eigenvalues λ and the corresponding eigenvectors x R n, that is, Ax = λx. While we have learnt in linear algebra results on the existence of the eigenvalues and corresponding eigenvectors, numerical linear algebra is concerned with the numerical computation of the eigenvalues on a computer for large square matrices A. The numerical solution of large linear systems and the numerical computation of eigenvalues are some of the most important topics in numerical analysis. For example, the approximation or interpolation of measured data, or the discretization of a differential equation lead to a linear system. The discretization of a differential equation can also lead to the problem of finding the eigenvalues of a matrix. We discuss two motivating examples that illustrate how the problem of the (numerical solution of a large linear system might arise in applications.. Motivation: An Interpolation Problem Suppose we are given N data sites a x < x <... < x N b in [a, b] and corresponding observations f, f,..., f N R. Suppose further that the observations follow an unknown iii

6 iv Introduction and Motivation generation process, that is, there is an unknown function f such that f(x i = f i. (For example, the data could be temperatures measured at a fixed time at equidistant locations along a thin metal rod. In this case we would like to use the measured temperature data to derive a function of the location along the thin metal rod that describes the temperature along the rod at the fixed time. One possibility to reconstruct the unknown function f is to choose a set of N continuous basis functions ϕ,...,ϕ N C([a, b] and to approximate f by a function s of the form s(x = N α j ϕ j (x, where the coefficients are determined by the interpolation conditions N f i = s(x i = α j ϕ j (x i, i =,,..., N. j= j= This leads to a linear system, which can be written in matrix form as ϕ (x ϕ (x... ϕ N (x α f ϕ (x ϕ (x... ϕ N (x α = f.. (. ϕ (x N ϕ (x N... ϕ N (x N The matrix in the linear system is an N N matrix, and if N is large, say N 6, then such a system is not easy to solve. The focus of this course is on how to solve linear systems, such as, for example, the one in the interpolation problem (. above. In this course, we will not discuss interpolation problems themselves, although they are a very interesting area of study and research. Despite that we make here two comments on the problem above. Remark. (comments on the interpolation problem (i One crucial issue is the choice of the basis functions ϕ, ϕ,..., ϕ N. For example, possible choices could be: (a polynomials: ϕ j (x = x j, j =,,..., N, or (b shifted Gaussians: ϕ j (x = e (x x j, j =,,..., N, where the x j in the definition of ϕ j is the data site x j. However, the choice of the basis functions is not arbitrary but will, in applications, be determined by information about the kind of measured data that is approximated. (ii Also, if the data f, f,...,f N is measured data then it is usually not exact but has measurement errors (noise, such that f i = f(x i + ǫ i, where ǫ i is the measurement error. In this case interpolation leads usually to a rather poor approximation of the data, and it would be better to use an approximation scheme that imposes conditions that demand only s(x i f i, i =,,..., N. α N f N

7 Introduction and Motivation v. Motivation: A Boundary Value Problem Consider the following one-dimensional boundary value problem: find a function u(x that is twice continuously differentiable and satisfies the differential equation subject to the boundary conditions The function f in (. is continuous on [, ]. d u dx(x + u(x = f(x on (,, (. u( = and u( =. (.3 One way to solve this boundary value problem numerically is to use finite differences. The basic idea here is to approximate the derivative by difference quotients: u (x u(x + h u(x h or u (x u(x u(x h. (.4 h The first formula in (.4 is a forward difference while the second one is a backward difference. Using first a forward and then a backward rule for the second derivative leads to the centred second difference u (x = u (x + h u (x h Finite Difference Method: ( u(x + h u(x h h u(x u(x h h u(x + h u(x + u(x h h. (.5 When finding a numerical approximation using finite differences we divide the interval [, ] into n+ subintervals of equal length h = /(n+ with endpoints at the equally spaced nodes x i = i h = i, i =,,..., n, n +. n + Our aim is to construct a vector u = u h = (u, u,...,u n, u n+ T such that u j is an approximation of u(x j, j =,,..., n, n +, where u denotes the (exact solution to the boundary value problem (. and (.3. Expressing (. and (.3 on the grid x, x,..., x n, x n+ and replacing the derivatives by finite differences, with the help of (.5, we obtain and u(x i+ u(x i + u(x i h + u(x i f(x i, i =,,..., n, (.6 u(x = and u(x n+ =. (.7

8 vi Introduction and Motivation Replacing in (.6 and (.7 the values u(x j by the approximations u j and using the abbreviation f i := f(x i, we get the equations and u i+ u i + u i h + u i = f i, i =,,..., n, (.8 u = and u n+ =. (.9 These equations (.8 and (.9 lead to the following linear system for the computation of the finite difference approximation u = u h = (u, u,...,u n, u n+ T : u + h u + h. f... u f. h =.. ( h u n f n + h u n f n... u n+ The involved matrix, which we denote by A, is in R (n+ (n+ and is tridiagonal. With f := (, f, f,...,f n, T, we can write (. as Au = f. Remark. (comments on the finite difference approximation (i This system of equations is sparse, that is, the number of non-zero entries is much less than (n+. This sparsity can be used to produce more efficient methods of storage, only storing the non-zero entries of the matrix. Also the sparse matrix reduces the number of required operations when calculating matrix-matrix and matrix-vector multiplications. (ii To obtain an accurate approximation to the true solution u we may have to choose h very small (very fine step size, thereby increasing the size of the linear system. (iii For a general boundary value problem in N-dimensions the size of the linear system can grow rapidly, for example, three dimensional problems grow over 8 times larger with each uniform refinement of the domain. In this course we will learn about direct methods (for example, Gaussian elimination and iterative methods (that is, the construction of a sequence of improving approximations to the solution that are used to numerically solve linear systems of equations (such as the ones encountered in this section and the previous section. We will look at how efficient (how much time and memory are required? and stable (do they give good approximations and do they converge and under what conditions? these methods are.

9 Chapter Revision: Some Linear Algebra In this chapter we first introduce common notation and give a brief revision of some definitions and results from linear algebra that we will frequently use in this course. In this course capital (upper-case letters A, B, C,... usually denote matrices, and bold-face lower-case letters a, b, x, y,... denote vectors. Functions are denoted by lower-case letters.. Vectors in R n and C n A vector x in R n (or C n is a column vector x x x =., where x, x,...,x n R (or x, x,...,x n C. x n The vector is the zero vector, where all entries are zero. We denote by e i in R n (or in C n the standard ith basis vector containing a one in the ith component and zeros elsewhere. For example in R 3 and C 3, the standard basis vectors are e =, e =, e 3 =. The vectors x,x,...,x m in R n (or in C n are linearly independent if the following holds true: If m a j x j = a x + a x a m x m = (. j=

10 .. Vectors in R n and C n with the real (complex numbers a, a,..., a m, then the numbers a j, j =,,..., m, are all zero. In other words, x,x,...,x m are linearly independent if the only real (complex numbers a, a,...,a m for which (. holds are a = a =... = a m =. The vectors x,x,...,x m in R n (or in C n are linearly dependent, if they are not linearly independent. This means x,x,...,x m in R n (or in C n are linearly dependent, if there exist real (complex numbers a, a,...,a m not all zero such that (. holds. Any m > n vectors in R n (in C n are linearly dependent. Any set of n linearly independent vectors in R n (in C n is a basis for R n (for C n. If v,v,...,v n is a basis for R n (for C n, then the following holds: For every vector x in R n (in C n, there exist uniquely determined real (complex numbers a, a,...,a n such that x = a j v j = a v + a v a n v n. x = j= For a column vector x in R n or in C n we denote by x T the transposed (row vector, that is, x x. x n and x T = (x, x,...,x n. Likewise the transpose of a row vector y is the corresponding column vector y T, that is, y y = (y, y,...,y n and y T y =.. For a column vector x C n we denote by x := x T the conjugate (row vector, that is, x x x =. x n and x = x T = y n (x, x,...,x n. Here, y indicates taking the complex conjugate of y C, that is, if y = a + i b with a, b R and i the imaginary unit, then y = a i b. Likewise the conjugate of a complex row vector y is the corresponding conjugate column vector y := y T, that is, y = (y, y,...,y n. and y = y T = y y. y n.

11 . Revision: Some Linear Algebra 3 For complex numbers y = a + i b C with a, b R, we have y = y y = (a i b(a + i b = a + b. The Euclidean inner product of two real-valued vectors x,y R n is given by x T y = x T y = x j y j = x y + x y x n y n. j= We note that the Euclidean inner product for R n is symmetric, that is, x T y = y T x for any x,y R n. The Euclidean inner product of two complex vectors x,y C n is given by x y = x y = x T y = x j y j = x y + x y x n y n. The Euclidean inner product for C n satisfies x y = y x for any x,y C n. j= j= The Euclidean norm of a vector x R n (or x C n is defined by x = ( / x T x = x j or x = ( / x x = x j. The geometric interpretation of the Euclidean norm x of a vector x in R n or C n is that x measures the length of x. We say that two vectors x and y in R n (or in C n are orthogonal (to each other if x T y = (or if x y =, respectively. For vectors in R n, this means geometrically that the angle between these two vectors is π/, that is 9. A set of m vectors x,x,...,x m in R n (or in C n is called orthogonal, if they are mutually orthogonal, that is, if x j is orthogonal to x k whenever j k. It is easily checked that a set of orthogonal vectors is, in particular, also linearly independent. A basis v,v,...,v n of R n is called an orthonormal basis for R n if the basis vectors have all length one and are mutually orthogonal, that is v j = for j =,,..., n, and v T j v k = for j, k =,..., n with j k. Likewise, a basis v,v,...,v n of C n is called an orthonormal basis for C n if the basis vectors have all length one and are mutually orthogonal, that is, v j = for j =,,..., n, and v j v k = for j, k =,..., n with j k. j= An orthonormal basis v,v,...,v n of R n has a very useful property: Any vector x R n has the representation x = (vj T xv j (. j=

12 4.. Matrices as a linear combination of the basis vectors v,v,...,v n. The validity of (. is easily established: Assume that x = n j= a j v j, and take the Euclidean inner product with v k. Because v,v,...,v n form an orthonormal basis, vk T v j = if j k and vk T v j = v k = if j = k. Thus ( vk T x = vk T a j v j = a j vk T v j = a k vk T v k = a k v k = a k. j= j= Replacing a j = v T j x in x = n j= a j v j now verifies (.. Analogously to (. an orthonormal basis v,v,...,v n of C n has the following property: Any vector x C n has the representation x = (vj xv j (.3 j= as a linear combination with respect to the orthonormal basis v,v,...,v n. Exercise State the properties of an inner product/scalar product for a complex vector space V, and verify that the Euclidean inner product for C n has these properties. Exercise Show formula (.3.. Matrices The matrix A C m n (or A R m n is an m n (m rows and n columns matrix with complex-valued (or real-valued entries: A := (a i,j = (a i,j i m; j n := where a i,j C (and a i,j R, respectively. a, a, a,n a, a, a,n... a m, a m, a m,n Occasionally, we will denote the column vectors of a matrix A = (a i,j in C m n (or in R m n by a j, j =,,..., n, that is A := (a,a,,a n, with a j = a,j a,j. a m,j, Cm (or R m, j =,,..., n.

13 . Revision: Some Linear Algebra 5 To denote the (i, jth entry a i,j of A = (a i,j we may occasionally also write A i,j := a i,j or A(i, j := a i,j. A matrix is called square if it has the same number of rows and columns. Thus square matrices are matrices in C n n or R n n. The diagonal of a square matrix A = (a i,j in C n n or in R n n are the entries a j,j, j =,,..., n. Vectors are special cases of matrices, and a column vector x C n (or x R n is just an n matrix. Likewise a row vector in C n (or R n is just an n matrix. Two matrices of special importance are the m n zero matrix, and, among the square matrices, the n n identity matrix: The zero matrix in C m n and in R m n is the m n matrix that has all entries zero. We denote the m n zero matrix by. The identity matrix in C n n and in R n n is the n n matrix in which the entries on the diagonal are all one and all other entries are zero. We denote the n n identity matrix by I. For example, in C 3 3 and R 3 3 we have = and I = The scalar multiplication of a matrix A = (a i,j in C m n (or in R m n with a complex (or real number µ is defined componentwise, that is, µ A in C m n (or in R m n, respectively is defined by (µ A i,j := µ a i,j, i =,,..., m; j =,,..., n. (.4 The addition of two m n matrices A = (a i,j and B = (b i,j in C m n (or in R m n is defined componentwise, that is, A + B in C m n (or in R m n, respectively is defined by (A + B i,j := a i,j + b i,j, i =,,..., m; j =,,..., n. (.5. The set C m n (or R m n of complex (or real m n matrices with the scalar multiplication (.4 and the addition (.5 forms a complex vector space (or real vector space, respectively. The matrix multiplication A B of A = (a i,j C m n and B = (b i,j C n p (or A = (a i,j R m n and B = (b i,j R n p gives the matrix C = (c i,j C m p (or C = (c i,j R m p, respectively, with the entries c i,j = a i,k b k,j = a i, b,j + a i, b,j a i,n b n,j, i =,,...,m; j =,,..., p. k= In words, c i,j is computed by taking the Euclidean inner product of the ith row vector of A with the jth column vector of B. Note that for square matrices A and B in C n n (or in R n n, both A B and B A are defined, but in general A B B A, that is, matrix multiplication is not commutative.

14 6.. Matrices Thus the Euclidean inner product x y of two vectors x and y in C n (and the Euclidean inner product x T y of two vectors x and y in R n is just the matrix multiplication of the n matrix x = x T (and x T, respectively with the n matrix y. The outer product B = (b i,j C n n of two vectors x,y C n is given by B = xy, where b i,j := x i y j, i =,,...n; j =,,...n. Analogously, the outer product of x and y in R n is xy T = (x i y j R n n. In analogy to the transpose of a vector in R n and the conjugate of a vector in C n, we define the transposed matrix of a matrix A R m n and the Hermitian conjugate matrix of a matrix A C n m. The transposed matrix or transpose of a matrix A = (a i,j in R m n (or in C m n is the matrix A T in R n m (or in C n m whose (i, jth entry is given by (A T i,j = a j,i. A square matrix A = (a i,j in R n n (or in C n n is called symmetric if A T = A, that is, a i,j = a j,i for all i, j =,,..., n. The Hermitian conjugate (matrix (or adjoint (matrix of A = (a i,j C m n is the matrix A := A T C n m whose (i, jthe entry is (A i,j = a j,i. A square matrix A = (a i,j C n n is called Hermitian (or self-adjoint if A = A, that is, a j,i = a i,j for all i, j =,,..., n. For A R m n and B R n p, we have and for A C m n and B C n p, we have (A B T = B T A T, (A B = B A. Let A = (a i,j C m n be a complex m n matrix. The null space or kernel of the matrix A is defined by null(a = ker(a := { x C n : Ax = } (.6 The range of the matrix A is defined by range(a := { y C m : Ax = y for some x C n}. (.7 The range of A is the linear space spanned by the column vectors of A.

15 . Revision: Some Linear Algebra 7 Analogous statements hold for real matrices A R m n with the only difference that C n and C m in (.6 and (.7 need to be replaced by R n and R m, respectively. The rank of a matrix A C m n or A R m n is the dimension of the range of A, that is, rank(a := dim ( range(a. The trace of a square matrix A = (a i,j C n n or A = (a i,j R n n, denoted by trace(a, is the sum of its diagonal entries, that is, trace(a = a j,j = a, + a, a n,n. j= Symmetric matrices in R n n and Hermitian matrices in C n n may have the following useful properties: A square matrix A = (a i,j R n n is called positive definite if A is symmetric (that is, A satisfies A T = A and x T Ax = a i,j x i x j > for all x R n with x. i= j= A square matrix A = (a i,j R n n is called positive semidefinite if A is symmetric and x T Ax = a i,j x i x j for all x R n. i= j= A square matrix A = (a i,j C n n is said to be positive definite if A is Hermitian (that is, A satisfies A = A and x Ax = a i,j x i x j > for all x C n with x. i= j= A square matrix A = (a i,j C n n is said to be positive semidefinite if A is Hermitian and x Ax = a i,j x i x j for all x C n. i= j= We have the following useful property of positive definite matrices: If A = (a i,j in R n n (or in C n n is positive definite, then the upper principal submatrices A p := (a i,j i,j p, p {,,..., n}, are positive definite, and det(a p > for all p {,,..., n}.

16 8.. Matrices Theorem. (characterization of positive definite matrices (i A symmetric matrix A R n n (that is, A = A T is positive definite if and only if all its eigenvalues are positive. (ii An Hermitian matrix A C n n (that is, A = A is positive definite if and only if all its eigenvalues are positive. Exercise 3 Show that C m n with the usual matrix addition and scalar multiplication is a complex vector space. Exercise 4 Find the range and the null space of the following matrix A := 3 5 Exercise 5 For a matrix A C m n show that the range of A is the linear space spanned by the column vectors of A.. Exercise 6 Which, if any, of the following square matrices are symmetric or Hermitian? (Note here i is always the imaginary unit! A := i i 3 i 3, B := i 7 5 i 5 3, C := Exercise 7 Show that (A B T = B T A T for any A R m n and B R n p. Show that (A B = B A for any A C m n and B C n p. Exercise 8 Compute the trace of the 3 3 matrix A = 3 3 3, (.8 Exercise 9 Show that the symmetric real 3 3 matrix A given by (.8 in the previous question is positive definite. Exercise Let A C n n be a positive definite matrix. If C C n m show that: (a C A C is positive semidefinite. (b rank ( C A C = rank(c. (c C A C is positive definite if and only if rank(c = m.

17 . Revision: Some Linear Algebra 9.3 Determinants of Square Matrices In this subsection let A be a square matrix with either real or complex entries. The determinant det(a of a matrix A = ( a b c d is defined by det(a = a b c d = a d bc. The determinant det(a of a 3 3 matrix A = a, a, a,3 a, a, a,3 a 3, a 3, a 3,3 is defined by det(a = a, a, a,3 a, a, a,3 a 3, a 3, a 3,3 = a, C, a, C, + a,3 C,3, where C,, C,, and C,3 are the so-called cofactors of a,, a,, and a,3, respectively, and are defined by C, = a, a,3 a 3, a 3,3, C, = a, a,3 a 3, a 3,3, and C,3 = a, a, a 3, a 3,. We observe that C,j is the determinant of the submatrix of A that is obtained by deleting the st row and jth column of A. The procedure for computing the determinant of a 3 3 matrix can be generalized to the following formula for computing the determinant of an n n matrix, where n : The determinant det(a of the n n matrix A = (a i,j is given by det(a = a i,j ( i+j C i,j, j= for any i {,,..., n}, where C i,j is the determinant of the (n (n submatrix of A obtained by deleting the ith row and jth column from A.

18 .4. Inverse Matrix of a Square Matrix Equivalently, we can also compute the determinant of an n n matrix A = (a i,j by expanding with respect to a column: det(a = a i,j ( i+j C i,j, i= for any j {,,..., n}, where C i,j is the determinant of the (n (n submatrix of A obtained by deleting the ith row and jth column from A. We note that for all n n matrices A, det(a = det(a T. Let A and B be n n matrices. Then the determinant of a product of two matrices satisfies det(a B = det(a det(b. Exercise Compute the determinant of the 3 3 matrix A = Exercise Prove the following statement: If A = (a i,j in R n n is positive definite, then the upper principal submatrices A p := (a i,j i,j p, p {,,..., n}, are positive definite. Exercise 3 Prove that for all square n n matrices A we have det(a = det(a T...4 Inverse Matrix of a Square Matrix A square matrix A in C n n (or in R n n is called invertible or non-singular, if there exists a matrix X in C n n (and in R n n, respectively such that A X = X A = I, where I is the n n identity matrix. The matrix X is then called the inverse (matrix of A, and we usually denote the inverse matrix of A by A. If a square matrix A in C n n or R n n is not invertible, then we call A singular. A fundamental result about inverse matrices is the following: A matrix square matrix A is invertible (or non-singular if and only if det(a. This implies conversely that a square matrix A is singular if and only if det(a =.

19 . Revision: Some Linear Algebra The easiest way to compute the inverse of a non-singular matrix A by hand is to write the augmented matrix (A I and then transform this system with elementary row operations such that we have the identity matrix on the left-hand side; then we obtain (I A with the inverse matrix A on the right-hand side. With the help of the inverse matrix we can solve linear systems as follows. Assume A is a non-singular square n n matrix in C n n and b is a given vector in C n. Then the linear system Ax = b has the solution x = A b, which follows easily from A b = A ( Ax = (A Ax = I x = x, where we have used the fact that matrix multiplication is associative. We will see in this course that computing the inverse of a large matrix is a rather expensive process, and therefore computing the inverse and then using x = A b is numerically not a good way to solve large linear systems. For two invertible (or non-singular square n n matrices A and B we have (A B = B A, and, for invertible matrices A R n n and B C n n, we have (A T = (A T and (B = (B. For a non-singular matrix A in C n n (or in R n n, we have det(a = ( det(a. A square matrix Q R n n is said to be orthogonal (or an orthogonal matrix if Q T = Q ( Q T Q = I and Q Q T = I. (.9 A square matrix Q C n n is said to be unitary (or a unitary matrix if Q = Q ( Q Q = I and Q Q = I. (. The second characterization in (.9 and (., respectively, tells us that the column vectors q i, i =,,...n, of Q are an orthonormal basis of R n and C n, respectively. That is, (.9 is equivalent to q T i q j = δ i,j, i, j =,,..., n, and (. is equivalent to q i q j = q i T q j = δ i,j, i, j =,,..., n, respectively, where δ i,j is the Kronecker delta, defined to be one if i = j and zero otherwise. Likewise, the row vectors of Q form an orthonormal basis of R n and C n, respectively.

20 .5. Eigenvalues and Eigenvectors of a Square Matrix Exercise 4 Show that the 3 3 matrix A = is non-singular. Compute the inverse matrix A of A. Exercise 5 Let A and B in C n n be invertible matrices. Prove the following statements: (a (A B = B A. (b (A = (A. Use this result to conclude that (A T = (A T if A is in R n n. (c det(a = (det(a. Exercise 6 Show that a positive definite matrix A R n n is non-singular and that the inverse matrix A is also positive definite. Exercise 7 Show that the inverse of a unitary matrix is unitary. Use this the result to show that the unitary matrices in C n n with the matrix multiplication form a (multiplicative group., Exercise 8 Consider the matrix A R 3 3 and the vector b R 3 given by A = 3 and b = 4. (a Compute the determinant of A. (b Is A invertible? Why? (c Compute the inverse matrix A of A (d Use the inverse matrix A to solve the linear system Ax = b. (e Show that A is positive definite. (Hint: use Theorem...5 Eigenvalues and Eigenvectors of a Square Matrix In this section, we consider R n n as a subset of C n n, so that all definitions for complex n n matrices also apply to real n n matrices. Let A be a square matrix in C n n. A complex number λ C is an eigenvalue of A if there exists a non-zero vector x C n \ {} such that Ax = λx. (.

21 . Revision: Some Linear Algebra 3 The vector x in (. is then called an eigenvector of A with the eigenvalue λ. By writing (. equivalently as λx Ax = (λ I Ax =, we see that a non-zero vector x satisfying (. exists if and only if det(λ I A =. The determinant p(a, λ := det(λ I A (. is a polynomial in λ of exact degree n, and (. is called the characteristic polynomial of A. Clearly, the (complex roots of the characteristic polynomial are the eigenvalues of the matrix A. By the fundamental theorem of algebra, any (complex polynomial of exact degree n has n complex roots, counted with multiplicity. Therefore, any square matrix A C n n has n complex eigenvalues, counted with multiplicity. To compute the eigenvalues and the corresponding eigenvectors of a square matrix A by hand, we proceed as follows: First we compute the characteristic polynomial p(a, λ = det(λ I A and find its roots. These roots are the eigenvalues of A. For each eigenvalue λ of A, we solve the linear system (λ I Ax = to find the eigenvectors x corresponding to λ. For a real n n matrix A R n n, the characteristic polynomial p(a, λ = det(λ I A has n complex roots (counted with multiplicity. In general, some (or even all of these roots may be not in R. Thus, for the special case A R n n, we can in general not conclude that A has n real eigenvalues, counted with multiplicity. It is clear that for large n the computation of the eigenvalues is a far from trivial problem..6 Other Notation: The Landau Symbol For two functions f, g : N R, we will write f = O(g if there is a constant C > and N N such that f(n C g(n for all n N. The symbol O is called the Landau symbol. For example, consider the matrix-vector multiplication Ax, where A C n n and x C n. Since the ith component of Ax is given by (Ax i = a i,j x j, (.3 j= the number of multiplications in (.3 is n and the number of additions in (.3 is n, so that the number of elementary operations to compute (Ax i is n, that is, O(n. The total

22 4.6. Other Notation: The Landau Symbol number of operations to compute a matrix-vector multiplication is therefore n(n = n n, that is, O(n. In numerical linear algebra the number of elementary operations needed to execute an algorithm is of great interest, since it determines the runtime and efficiency of the algorithm. Usually information about the cost or number of elementary operations is not given as an exact figure but rather by listing it as O(n, O(n, O(n 3,..., as appropriate, in terms of the dimension n of the problem. In the example above the dimension is the number of components n of the involved vectors.

23 Chapter Matrix Theory In this chapter we learn various basics from matrix theory and encounter the first numerical algorithm. In Section., we revise some facts and results about the eigenvalues and eigenvectors of a square matrix in C n n. These facts are needed throughout the course, and we will come back to them at various stages later-on. In Section., we introduce upper triangular matrices (and lower triangular matrices, and we learn how a linear system with an upper triangular matrix can be easily solved by back substitution. In Section.3, we learn about the Schur factorization: Given a square matrix A in C n n, the Schur factorization guarantees that there exists a unitary matrix S, such that the matrix U = S A S = S A S is an upper triangular matrix. This result will be exploited at various stages later in the course. In Section.4, we revise some facts about vector norms, and in Section.5, we introduce a variety of matrix norms that will be used throughout the course. In Section.6, we define the spectral radius of a square matrix A C n n which is the maximum of the absolute values of the n complex eigenvalues of A. In formulas, let λ, λ,...,λ n C be the n complex eigenvalues (counted with multiplicity of A C n n ; then the spectral radius of A C n n is defined by ρ(a := max { λ, λ,..., λ n }. We also learn a result about the relation between the spectral radius and matrix norms.. The Eigensystem of a Square Matrix The material in this section on the eigensystem, that is, the eigenvalues and eigenvectors, of a square matrix should be familiar from linear algebra. However, it is strongly recommended that you carefully go through this section, revise the material, and solve the exercises! 5

24 6.. The Eigensystem of a Square Matrix In this section real square matrices A R n n are considered as the special case of matrices in C n n with real entries; so all results for matrices in C n n hold also for matrices in R n n. Consider a square matrix A in C n n, A = (a ij = a, a, a,n a, a, a,n a n, a n, a n,n. Definition. (eigenvalues and eigenvectors A complex number λ C is called an eigenvalue of A = (a i,j C n n if there exists a non-zero vector x C n such that Ax = λx λx Ax = ( λ I A x =, (. where I is the n n identity matrix. A vector x satisfying (. is called an eigenvector corresponding to the eigenvalue λ. From (. it is clear that, once we know an eigenvalue λ, any corresponding eigenvector x can be computed by solving the linear system (λ I Ax =. The linear system (λ I Ax = has non-zero solutions if and only if det(λ I A =. Thus we have found a criterion for determining whether a complex number is an eigenvalue: λ is an eigenvalue of A if and only if det(λ I A =. From the properties of the determinant it is easily seen that det(λ I A = λ n + c n λ n c λ + c, (. with suitable complex coefficients c, c,...,c n, that is, det(λ I A is a polynomial in λ of exact degree n. Thus any eigenvalue of A is a root of the polynomial det(λ I A. Theorem. (roots of the characteristic polynomial are eigenvalues Let A C n n. A complex number λ C is an eigenvalue of A if and only if it is a root of the characteristic polynomial of A, defined by p(a, λ := det(λ I A. Counted with multiplicity, the characteristic polynomial has exactly n complex roots, that is, A has exactly n complex eigenvalues, counted with multiplicity. The last statement in the theorem above follows from the fundamental theorem of algebra. Definition.3 (spectrum of a matrix Let A C n n. The set of all eigenvalues of A is called the spectrum of A and is denoted by Λ(A := { λ C : det(λ I A = }.

25 . Matrix Theory 7 Once we have found the n complex eigenvalues of A C n n, we can compute the corresponding eigenvectors x to an eigenvalue λ by solving the linear system (λ I A x =. If an eigenvalue occurs with a multiplicity k >, there can be as at most k linearly independent eigenvectors. It is not difficult to show that the set of eigenvectors to a given eigenvalue λ forms a vector space, called the eigenspace of the eigenvalue λ. Indeed let λ be an eigenvalue of A and define the eigenspace of λ by E λ (A := {x C n : Ax = λx}. Now consider two elements x and y from E λ (A. Then the linear combination αx+β y is also in E λ (A, since A ( αx + β y = α Ax + β Ay = α λx + β λy = λ ( αx + β y. This guarantees closure under vector addition and scalar multiplication, and hence E λ C n is a vector space. Example.4 (eigenvalues and eigenvectors Compute the eigenvalues and eigenvectors of the matrix A = Solution: We compute the roots of the characteristic polynomial λ det(λ I A = det λ λ 5. = λ (λ + 3 (λ (λ λ 4 (λ 5 = λ 3 λ λ + = (λ (λ (λ +, where the roots were determined by guessing that λ = is a root and then using long division and the binomial formulas. Thus we have found that λ =, λ = and λ 3 = are the eigenvalues of A. To compute the corresponding eigenvectors x j, we solve for each eigenvalue λ j the linear system (λ j I Ax j =, which we write in augmented matrix form as (λ j I A and solve by Gaussian elimination. For λ =, we have ( I Ax = x =

26 8.. The Eigensystem of a Square Matrix We multiply the first row of the augmented matrix by /, and in the next step, we add ( times the new first row to the second row and ( 3 times the new first row to the third row Now we subtract the new second row from the new third row, and subsequently we divide the new second row by Finally, we subtract the second row from the first row and obtain x = x = α, where α R. Thus all eigenvectors x corresponding to λ = are of the form x = α (,, T, where α R \ {}. For λ =, we have (I Ax = x = We multiply the first row by ( and add it to the second row, and we multiply the first row by ( 3 and add it to the third row. Subsequently we subtract the new second row from the new third row. Then we divide the new second row by. Finally we add times the second row to the first row. Thus x = x = α with α R. Thus all eigenvectors corresponding to the eigenvalue λ = are of the form x = α (,, T with α R \ {}.,.

27 . Matrix Theory 9 For λ 3 =, we have ( I Ax = x = We multiply the first row by and add it to the second row, and we multiply the first row by 3 and add it to the third row. Subsequently we multiply the new second row by ( and add it to the third row. Afterwards we divide the new second row by Finally we add times the second row to the first row multiplied by (. x 3 = x 3 = α where α R. Thus all eigenvectors corresponding to the eigenvalue λ 3 = are of the form x 3 = α (,, T, with α R \ {}. We summarize what we have found so far: the spectrum of A is and the eigenspaces E λ (A of the eigenvalues λ are., Λ(A = {,, }, (.3 E = { α (,, T : α R }, E = { α (,, T : α R }, E = { α (,, T : α R }. (.4 This completes the example. Exercise 9 Compute the eigenvalues and corresponding eigenvectors of the 3 3 matrix A = Exercise Consider the matrix A R 3 3 given by A =. 3

28 .. The Eigensystem of a Square Matrix (a Compute the eigenvalues of A by hand. (b Compute all eigenvectors to the eigenvalue that is an integer by hand. An important property of eigenvalues is that they are invariant under so-called basis transformations or similarity transformations. Lemma.5 (eigenvalues are invariant under basis transformation Let A C n n, and let S C n n with det(s. Then S A S is called a basis transformation or similarity transformation of A, and det(λ I S A S = det(λ I A, so that A and S AS have the same eigenvalues. We say the eigenvalues of A are invariant under a basis transformation or a similarity transformation. Proof of Lemma.5. From det(b C = det(b det(c and S I S = S S = I, det(λ I S A S = det ( S (λ I A S = det(s det(λ I A det(s, and, since det(s = (det(s, the result follows. Lemma.5 gives us the following idea: If we are only interested in the eigenvalues of a square matrix A C n n, then it would be useful to find a suitable non-singular matrix S C n n, such that the eigenvalues of S A S are easier to compute. In order to execute this idea, we first need to understand the nature of eigenvectors better. The following lemma is elementary but has far reaching consequences. Lemma.6 (eigenvectors to different eigenvalues are linearly independent Let A C n n. Eigenvectors to different eigenvalues of A are linearly independent. More precisely, let λ, λ,...,λ m be m distinct eigenvalues of A, and let x,x,...,x m be corresponding eigenvectors, that is, Ax j = λ j x j for j =,,..., m. Then the eigenvectors x,x,...,x m are linearly independent. Proof of Lemma.6. The proof is given by induction on m. Initial step m = : Consider two different eigenvalues λ and λ and let x and x be two corresponding eigenvectors, that is, Ax i = λ i x i, i =,. To show the linear independence of x and x consider α x + α x =. (.5 If (.5 implies α = α =, then we have shown that x and x are linearly independent. Assume therefore that at least one of the coefficients α and α in (.5 is non-zero, say α.

29 . Matrix Theory Then from (.5 Multiplying from the left with A on both sides of (.6 yields x = α α x. (.6 Ax = α α Ax λ x = α α λ x = λ x (λ λ x =, (.7 where we have used (.6 in the middle equation. Since λ λ and x, we have (λ λ x, and the last formula in (.7 is a contradiction. We see that only α = α = are possible in (.5, and hence x and x are linearly independent. Induction step m m + : The induction step is left as an exercise. Exercise Show the induction step in the proof of Lemma.6. After these preparations we can present one of the major theorems of linear algebra. Theorem.7 (basis transformation into diagonal form Let A C n n, and let λ, λ,..., λ n C denote its n complex eigenvalues. If there are n linearly independent corresponding eigenvectors x,x,...,x n (that is, Ax j = λ j x j for j =,,..., n, then the n eigenvectors x,x,...,x n form a basis of C n. Under this assumption, let S denote the matrix that contains the eigenvectors x,x,...,x n as column vectors, that is, S := (x,x,...,x n. Then the basis transformation (or similarity transformation S A S yields the diagonal matrix with the eigenvalues λ, λ,...,λ n along the diagonal. In formulas, S A S = λ λ. λ λ n. The proof of this theorem is surprisingly simple and very intuitive, and greatly helps in understanding Theorem.7. Proof of Theorem.7. We first consider A S. Since the jth column vector of S = (s i,j is the eigenvector x j (that is, x j = (s,j, s,j,...,s n,j T and Ax j = λ j x j, we have that (A S k,j = a k,i s i,j, = λ j s k,j, k =,,..., n. (.8 i=

30 .. The Eigensystem of a Square Matrix In other words, A S is the matrix whose jth column is given by λ j x j. The (i, jth entry in S A S = S (A S is given by (S A S i,j = (S i,k (A S k,j = (S i,k λ j s k,j = λ j (S i,k s k,j = λ j δ i,j, k= k= where we have used in the last step that S S = I. Thus S A S is indeed the diagonal matrix with the eigenvalues λ, λ,...,λ n along the diagonal. Example.8 We illustrate Theorem.7 for our matrix from Example.4 A = In Example.4, we found that the spectrum of A is (see (.3 Λ(A = {,, }, and that the eigenspaces of the eigenvalues are (see (.4 Hence we choose the matrix S to be and we expect that E = { α (,, T : α R }, E = { α (,, T : α R }, E = { α (,, T : α R }. S = S A S =, k=. (.9 To verify (.9, we compute S and then execute the matrix multiplication S A S. We write the augmented linear system (S I, and use elementary row operations to transform it into (I S, and we find (details left as an exercise. Thus the inverse S is given by S =,

31 . Matrix Theory 3 and executing the matrix multiplications in (.9 shows that (.9 is indeed true. Note that in the definition of the matrix S the normalization of the basis vectors plays no role. That is, if we choose instead of S T = α α α α 3 α α 3, with any non-zero numbers α, α, α 3 R, the we also have T A T = The permutation of the columns in S or T corresponds to a corresponding permutation of the eigenvalues in the diagonal matrix. Exercise For the matrix A from Example 9, A = 3 3 3,. find a matrix S such that S A S = λ λ λ 3, with λ λ λ 3. Compute S and execute the matrix multiplication S A S to verify that you have chosen S correctly. Whether a matrix A C n n has n linearly independent eigenvectors is a non-trivial problem. The following lemma gives a sufficient but not a necessary condition for the existence of n linearly independent eigenvalues. Lemma.9 (sufficient cond. for the existence of n lin. indep. eigenvectors Let A C n n have n distinct complex eigenvalues λ, λ,...,λ n (that is, the eigenvalues λ, λ,...,λ n are all different. Then A has n linearly independent eigenvectors. Proof of Lemma.9. For each eigenvalue λ j choose one eigenvector x j. From Lemma.6, the eigenvectors x,x,...,x n are linearly independent, since the eigenvalues λ, λ,...,λ n are distinct.

32 4.. The Eigensystem of a Square Matrix From det(s A S = det(s det(a det(s = (det(s det(a det(s = det(a it is clear that the determinant is invariant under a basis transformation. Thus if λ, λ,...,λ n are the n complex eigenvalues of A and if A has n linearly independent eigenvectors, then Theorem.7 implies that det(a = λ λ λ n. (. We mention here that the trace of the square matrix (defined as the sum of its diagonal elements is also invariant under basis transformations, that is, trace(s A S = trace(a. Hence, we see with the help of Theorem.7 that if A C n n has n linearly independent eigenvectors, then the trace of A is given by where λ, λ n,..., λ n denote the n complex eigenvalues of A. trace(a = λ + λ + + λ n, (. In fact, (. and (. even hold without the assumption that A has n linearly independent eigenvectors, but the proof of this needs more advanced linear algebra. Later-on we need the following special cases of Theorem.7 above, where A R n n is symmetric or A C n n is Hermitian. For these results, remember the following definitions: A matrix S C n n is called unitary if S = S T = S, and a matrix S R n n is called orthogonal if S T = S. Theorem. (basis transformation into diagonal form for Hermitian matrices Let A C n n be Hermitian, that is, A = A. Then there exists a unitary matrix S C n n such that λ λ S A S = S A S =. λ λ n The values λ, λ,...,λ n are real and are the eigenvalues of A, and the columns of the matrix S are an orthonormal basis of eigenvectors. More precisely, if we denote the jth column vector of S by x j, j =,,..., n, then x,x,...,x n is an orthonormal basis of C n and Ax j = λ j x j, j =,,..., n. For the special case of real matrices, we have the following result.

33 . Matrix Theory 5 Theorem. (basis transformation into diagonal form for symmetric matrices Let A R n n be symmetric, that is, A = A T. Then there exists an orthogonal matrix S R n n such that λ λ S T A S = S A S = λ λ n The values λ, λ,...,λ n are real and are the eigenvalues of A, and the columns of the matrix S are an orthonormal basis of eigenvectors. More precisely, if we denote the jth column vector of S by x j, j =,,..., n, then x,x,...,x n is an orthonormal basis of R n and Ax j = λ j x j, j =,,..., n. Proof of Theorem.. A proof of Theorem. will be discussed in Exercise 3 with the help of the Schur factorization. Exercise 3 For the matrix A from Examples 9 and, A = 3 3 3, find an orthogonal matrix S such that S A S = λ λ λ 3, (. with λ λ λ 3. Verify that your matrix S is orthogonal. Verify that (. is true by executing the matrix multiplications in (.. Exercise 4 Let A C n n, and let S C n n be a non-singular matrix. Show that trace(s A S = trace(a. Exercise 5 Consider the real matrix ( 3 A = 3. (a Calculate the eigenvalues λ and λ (where λ λ of A by hand.

34 6.. Upper Triangular Matrices and Back Substitution (b Calculate the eigenspaces correponding to the eigenvalues from (a by hand. (c Find an orthogonal matrix S (that is, S T = S such that ( S T λ A S = λ, where λ > λ. Execute the matrix multiplication S T A S to verify that your choice of S is correct.. Upper Triangular Matrices and Back Substitution Two important classes of matrices are upper triangular matrices and lower triangular matrices. Definition. (upper and lower triangular matrices A square matrix A = (a i,j in C n n or R n n is said to be upper triangular (or an upper triangular matrix if a i,j = for i > j. Thus an n n upper triangular matrix is of the form a, a, a,3 a,n a, a,3 a,n A = (a i,j = a 3,3 a 3,n. ( a n,n Similarly, an n n matrix A = (a i,j is said to be lower triangular (or a lower triangular matrix if a i,j = for i < j. Example.3 (upper and lower triangular matrices A = 3, B = Then A is a 3 3 upper triangular matrix, and B is a 3 3 lower triangular matrix. The following lemma establishes some important properties of lower triangular and upper triangular matrices.

35 . Matrix Theory 7 Lemma.4 (properties of upper/lower triangular matrices The set of upper triangular matrices in C n n (or R n n with the usual matrix addition and usual scalar multiplication with complex (or real numbers forms a complex (or real vector space. Let A, B C n n (or A, B R n n be upper triangular matrices. Then A B is also an upper triangular matrix in C n n (or R n n, respectively. If A is non-singular, then A is also an upper triangular matrix in C n n (or in R n n, respectively. Analogous statements hold for lower triangular matrices. Let A = (a i,j in C n n or in R n n be an upper triangular or lower triangular matrix. Then the following holds: (i det(a = a, a, a n,n. (ii A is non-singular/invertible if and only if a j,j for all j =,,..., n. (iii The eigenvalues of A are the entries a,, a,,...,a n,n on the diagonal of A. Proof of Lemma.4. The set of all n n matrices in C n n (or R n n with matrix addition and scalar multiplication with complex (or real numbers forms a complex (or real vector space. Therefore, to verify that the upper triangular matrices form a vector space, it is enough to check the closure under addition and scalar multiplication. This is easily done and is left as an exercise. The next statements are covered in the exercises, but we prove (i to (iii for upper triangular matrices. Let A be an n n upper triangular matrix (.3. Computing the determinant det(a by expansion with respect to the first column yields a, a,3 a,n a 3,3 a 3,n det(a = a, det a n,n The resulting submatrix whose determinant needs to be computed is again upper triangular, and repeating the procedure yields finally det(a = a, a, a n,n, (.4 which proves (i. Since a matrix is invertible if and only if det(a, (.4 implies immediately (ii. To see the statement (iii consider the matrix λ I A. Since A is upper triangular and λ I is diagonal, the matrix λ I A is also upper triangular and its entries on the diagonal are λ a j,j, j =,,..., n. Thus form (.4, p(a, λ = det ( λ I A = (λ a (λ a, (λ a n,n, and we see that the eigenvalues of A are indeed a,, a,,...,a n,n.

Foundations of Matrix Analysis

Foundations of Matrix Analysis 1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the

More information

Review problems for MA 54, Fall 2004.

Review problems for MA 54, Fall 2004. Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on

More information

MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, Chapter 1: Linear Equations and Matrices MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

More information

Lecture Summaries for Linear Algebra M51A

Lecture Summaries for Linear Algebra M51A These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture

More information

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0. Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the

More information

Fundamentals of Engineering Analysis (650163)

Fundamentals of Engineering Analysis (650163) Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is

More information

CS 246 Review of Linear Algebra 01/17/19

CS 246 Review of Linear Algebra 01/17/19 1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

More information

A Review of Matrix Analysis

A Review of Matrix Analysis Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value

More information

Numerical Methods - Numerical Linear Algebra

Numerical Methods - Numerical Linear Algebra Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear

More information

Jim Lambers MAT 610 Summer Session Lecture 1 Notes

Jim Lambers MAT 610 Summer Session Lecture 1 Notes Jim Lambers MAT 60 Summer Session 2009-0 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra

More information

EE731 Lecture Notes: Matrix Computations for Signal Processing

EE731 Lecture Notes: Matrix Computations for Signal Processing EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten

More information

Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in

Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column

More information

Linear Algebra Review

Linear Algebra Review Chapter 1 Linear Algebra Review It is assumed that you have had a course in linear algebra, and are familiar with matrix multiplication, eigenvectors, etc. I will review some of these terms here, but quite

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2 MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

1 Matrices and Systems of Linear Equations

1 Matrices and Systems of Linear Equations March 3, 203 6-6. Systems of Linear Equations Matrices and Systems of Linear Equations An m n matrix is an array A = a ij of the form a a n a 2 a 2n... a m a mn where each a ij is a real or complex number.

More information

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88 Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

More information

1 Matrices and Systems of Linear Equations. a 1n a 2n

1 Matrices and Systems of Linear Equations. a 1n a 2n March 31, 2013 16-1 16. Systems of Linear Equations 1 Matrices and Systems of Linear Equations An m n matrix is an array A = (a ij ) of the form a 11 a 21 a m1 a 1n a 2n... a mn where each a ij is a real

More information

Linear Algebra M1 - FIB. Contents: 5. Matrices, systems of linear equations and determinants 6. Vector space 7. Linear maps 8.

Linear Algebra M1 - FIB. Contents: 5. Matrices, systems of linear equations and determinants 6. Vector space 7. Linear maps 8. Linear Algebra M1 - FIB Contents: 5 Matrices, systems of linear equations and determinants 6 Vector space 7 Linear maps 8 Diagonalization Anna de Mier Montserrat Maureso Dept Matemàtica Aplicada II Translation:

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 1: Course Overview; Matrix Multiplication Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical

More information

Scientific Computing: Dense Linear Systems

Scientific Computing: Dense Linear Systems Scientific Computing: Dense Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 February 9th, 2012 A. Donev (Courant Institute)

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 1: Course Overview & Matrix-Vector Multiplication Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 20 Outline 1 Course

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017 Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...

More information

1. General Vector Spaces

1. General Vector Spaces 1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

More information

MAT 610: Numerical Linear Algebra. James V. Lambers

MAT 610: Numerical Linear Algebra. James V. Lambers MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic

More information

Linear Systems and Matrices

Linear Systems and Matrices Department of Mathematics The Chinese University of Hong Kong 1 System of m linear equations in n unknowns (linear system) a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.......

More information

j=1 x j p, if 1 p <, x i ξ : x i < ξ} 0 as p.

j=1 x j p, if 1 p <, x i ξ : x i < ξ} 0 as p. LINEAR ALGEBRA Fall 203 The final exam Almost all of the problems solved Exercise Let (V, ) be a normed vector space. Prove x y x y for all x, y V. Everybody knows how to do this! Exercise 2 If V is a

More information

Linear Algebra Primer

Linear Algebra Primer Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

More information

LINEAR ALGEBRA REVIEW

LINEAR ALGEBRA REVIEW LINEAR ALGEBRA REVIEW SPENCER BECKER-KAHN Basic Definitions Domain and Codomain. Let f : X Y be any function. This notation means that X is the domain of f and Y is the codomain of f. This means that for

More information

Linear Algebra Highlights

Linear Algebra Highlights Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to

More information

MTH 464: Computational Linear Algebra

MTH 464: Computational Linear Algebra MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University March 2, 2018 Linear Algebra (MTH 464)

More information

Cheat Sheet for MATH461

Cheat Sheet for MATH461 Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A

More information

Extra Problems for Math 2050 Linear Algebra I

Extra Problems for Math 2050 Linear Algebra I Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as

More information

1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )

1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i ) Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical

More information

2 b 3 b 4. c c 2 c 3 c 4

2 b 3 b 4. c c 2 c 3 c 4 OHSx XM511 Linear Algebra: Multiple Choice Questions for Chapter 4 a a 2 a 3 a 4 b b 1. What is the determinant of 2 b 3 b 4 c c 2 c 3 c 4? d d 2 d 3 d 4 (a) abcd (b) abcd(a b)(b c)(c d)(d a) (c) abcd(a

More information

1 Last time: least-squares problems

1 Last time: least-squares problems MATH Linear algebra (Fall 07) Lecture Last time: least-squares problems Definition. If A is an m n matrix and b R m, then a least-squares solution to the linear system Ax = b is a vector x R n such that

More information

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v ) Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

More information

Linear Algebra Primer

Linear Algebra Primer Introduction Linear Algebra Primer Daniel S. Stutts, Ph.D. Original Edition: 2/99 Current Edition: 4//4 This primer was written to provide a brief overview of the main concepts and methods in elementary

More information

Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.

Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same. Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read

More information

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2

More information

Linear Algebra: Matrix Eigenvalue Problems

Linear Algebra: Matrix Eigenvalue Problems CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given

More information

Algebra C Numerical Linear Algebra Sample Exam Problems

Algebra C Numerical Linear Algebra Sample Exam Problems Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric

More information

Knowledge Discovery and Data Mining 1 (VO) ( )

Knowledge Discovery and Data Mining 1 (VO) ( ) Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory

More information

OHSx XM511 Linear Algebra: Solutions to Online True/False Exercises

OHSx XM511 Linear Algebra: Solutions to Online True/False Exercises This document gives the solutions to all of the online exercises for OHSx XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Answers are in square brackets [. Lecture 02 ( 1.1)

More information

Numerical Linear Algebra

Numerical Linear Algebra Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an n-vector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix

More information

Solution of Linear Equations

Solution of Linear Equations Solution of Linear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass

More information

Math Linear Algebra Final Exam Review Sheet

Math Linear Algebra Final Exam Review Sheet Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

More information

MAT Linear Algebra Collection of sample exams

MAT Linear Algebra Collection of sample exams MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

3 Matrix Algebra. 3.1 Operations on matrices

3 Matrix Algebra. 3.1 Operations on matrices 3 Matrix Algebra A matrix is a rectangular array of numbers; it is of size m n if it has m rows and n columns. A 1 n matrix is a row vector; an m 1 matrix is a column vector. For example: 1 5 3 5 3 5 8

More information

Elementary maths for GMT

Elementary maths for GMT Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

More information

Linear Algebra and Matrix Inversion

Linear Algebra and Matrix Inversion Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much

More information

5.6. PSEUDOINVERSES 101. A H w.

5.6. PSEUDOINVERSES 101. A H w. 5.6. PSEUDOINVERSES 0 Corollary 5.6.4. If A is a matrix such that A H A is invertible, then the least-squares solution to Av = w is v = A H A ) A H w. The matrix A H A ) A H is the left inverse of A and

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

Topic 1: Matrix diagonalization

Topic 1: Matrix diagonalization Topic : Matrix diagonalization Review of Matrices and Determinants Definition A matrix is a rectangular array of real numbers a a a m a A = a a m a n a n a nm The matrix is said to be of order n m if it

More information

Matrix Factorization and Analysis

Matrix Factorization and Analysis Chapter 7 Matrix Factorization and Analysis Matrix factorizations are an important part of the practice and analysis of signal processing. They are at the heart of many signal-processing algorithms. Their

More information

(f + g)(s) = f(s) + g(s) for f, g V, s S (cf)(s) = cf(s) for c F, f V, s S

(f + g)(s) = f(s) + g(s) for f, g V, s S (cf)(s) = cf(s) for c F, f V, s S 1 Vector spaces 1.1 Definition (Vector space) Let V be a set with a binary operation +, F a field, and (c, v) cv be a mapping from F V into V. Then V is called a vector space over F (or a linear space

More information

Math113: Linear Algebra. Beifang Chen

Math113: Linear Algebra. Beifang Chen Math3: Linear Algebra Beifang Chen Spring 26 Contents Systems of Linear Equations 3 Systems of Linear Equations 3 Linear Systems 3 2 Geometric Interpretation 3 3 Matrices of Linear Systems 4 4 Elementary

More information

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n Lectures -2: Linear Algebra Background Almost all linear and nonlinear problems in scientific computation require the use of linear algebra These lectures review basic concepts in a way that has proven

More information

Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

More information

Practical Linear Algebra: A Geometry Toolbox

Practical Linear Algebra: A Geometry Toolbox Practical Linear Algebra: A Geometry Toolbox Third edition Chapter 12: Gauss for Linear Systems Gerald Farin & Dianne Hansford CRC Press, Taylor & Francis Group, An A K Peters Book www.farinhansford.com/books/pla

More information

A Brief Outline of Math 355

A Brief Outline of Math 355 A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting

More information

Eigenvalues and eigenvectors

Eigenvalues and eigenvectors Chapter 6 Eigenvalues and eigenvectors An eigenvalue of a square matrix represents the linear operator as a scaling of the associated eigenvector, and the action of certain matrices on general vectors

More information

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear

More information

Digital Workbook for GRA 6035 Mathematics

Digital Workbook for GRA 6035 Mathematics Eivind Eriksen Digital Workbook for GRA 6035 Mathematics November 10, 2014 BI Norwegian Business School Contents Part I Lectures in GRA6035 Mathematics 1 Linear Systems and Gaussian Elimination........................

More information

Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

More information

Chapter 5 Eigenvalues and Eigenvectors

Chapter 5 Eigenvalues and Eigenvectors Chapter 5 Eigenvalues and Eigenvectors Outline 5.1 Eigenvalues and Eigenvectors 5.2 Diagonalization 5.3 Complex Vector Spaces 2 5.1 Eigenvalues and Eigenvectors Eigenvalue and Eigenvector If A is a n n

More information

NOTES on LINEAR ALGEBRA 1

NOTES on LINEAR ALGEBRA 1 School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

More information

This can be accomplished by left matrix multiplication as follows: I

This can be accomplished by left matrix multiplication as follows: I 1 Numerical Linear Algebra 11 The LU Factorization Recall from linear algebra that Gaussian elimination is a method for solving linear systems of the form Ax = b, where A R m n and bran(a) In this method

More information

B553 Lecture 5: Matrix Algebra Review

B553 Lecture 5: Matrix Algebra Review B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations

More information

Matrices A brief introduction

Matrices A brief introduction Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 44 Definitions Definition A matrix is a set of N real or complex

More information

MATH 583A REVIEW SESSION #1

MATH 583A REVIEW SESSION #1 MATH 583A REVIEW SESSION #1 BOJAN DURICKOVIC 1. Vector Spaces Very quick review of the basic linear algebra concepts (see any linear algebra textbook): (finite dimensional) vector space (or linear space),

More information

Introduction to Matrices

Introduction to Matrices 214 Analysis and Design of Feedback Control Systems Introduction to Matrices Derek Rowell October 2002 Modern system dynamics is based upon a matrix representation of the dynamic equations governing the

More information

Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors Chapter 1 Eigenvalues and Eigenvectors Among problems in numerical linear algebra, the determination of the eigenvalues and eigenvectors of matrices is second in importance only to the solution of linear

More information

Notes on Mathematics

Notes on Mathematics Notes on Mathematics - 12 1 Peeyush Chandra, A. K. Lal, V. Raghavendra, G. Santhanam 1 Supported by a grant from MHRD 2 Contents I Linear Algebra 7 1 Matrices 9 1.1 Definition of a Matrix......................................

More information

Review Questions REVIEW QUESTIONS 71

Review Questions REVIEW QUESTIONS 71 REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of

More information

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the

More information

A Review of Linear Algebra

A Review of Linear Algebra A Review of Linear Algebra Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab: Implementations

More information

Calculating determinants for larger matrices

Calculating determinants for larger matrices Day 26 Calculating determinants for larger matrices We now proceed to define det A for n n matrices A As before, we are looking for a function of A that satisfies the product formula det(ab) = det A det

More information

Matrices. In this chapter: matrices, determinants. inverse matrix

Matrices. In this chapter: matrices, determinants. inverse matrix Matrices In this chapter: matrices, determinants inverse matrix 1 1.1 Matrices A matrix is a retangular array of numbers. Rows: horizontal lines. A = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 a 41 a

More information

Lecture Notes for Inf-Mat 3350/4350, Tom Lyche

Lecture Notes for Inf-Mat 3350/4350, Tom Lyche Lecture Notes for Inf-Mat 3350/4350, 2007 Tom Lyche August 5, 2007 2 Contents Preface vii I A Review of Linear Algebra 1 1 Introduction 3 1.1 Notation............................... 3 2 Vectors 5 2.1 Vector

More information

MAT 2037 LINEAR ALGEBRA I web:

MAT 2037 LINEAR ALGEBRA I web: MAT 237 LINEAR ALGEBRA I 2625 Dokuz Eylül University, Faculty of Science, Department of Mathematics web: Instructor: Engin Mermut http://kisideuedutr/enginmermut/ HOMEWORK 2 MATRIX ALGEBRA Textbook: Linear

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

Spectral Theorem for Self-adjoint Linear Operators

Spectral Theorem for Self-adjoint Linear Operators Notes for the undergraduate lecture by David Adams. (These are the notes I would write if I was teaching a course on this topic. I have included more material than I will cover in the 45 minute lecture;

More information

MATHEMATICS 217 NOTES

MATHEMATICS 217 NOTES MATHEMATICS 27 NOTES PART I THE JORDAN CANONICAL FORM The characteristic polynomial of an n n matrix A is the polynomial χ A (λ) = det(λi A), a monic polynomial of degree n; a monic polynomial in the variable

More information

Matrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices

Matrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices Matrices A. Fabretti Mathematics 2 A.Y. 2015/2016 Table of contents Matrix Algebra Determinant Inverse Matrix Introduction A matrix is a rectangular array of numbers. The size of a matrix is indicated

More information

Chapter 2:Determinants. Section 2.1: Determinants by cofactor expansion

Chapter 2:Determinants. Section 2.1: Determinants by cofactor expansion Chapter 2:Determinants Section 2.1: Determinants by cofactor expansion [ ] a b Recall: The 2 2 matrix is invertible if ad bc 0. The c d ([ ]) a b function f = ad bc is called the determinant and it associates

More information

Linear Algebra: Lecture Notes. Dr Rachel Quinlan School of Mathematics, Statistics and Applied Mathematics NUI Galway

Linear Algebra: Lecture Notes. Dr Rachel Quinlan School of Mathematics, Statistics and Applied Mathematics NUI Galway Linear Algebra: Lecture Notes Dr Rachel Quinlan School of Mathematics, Statistics and Applied Mathematics NUI Galway November 6, 23 Contents Systems of Linear Equations 2 Introduction 2 2 Elementary Row

More information

Eigenvalue and Eigenvector Problems

Eigenvalue and Eigenvector Problems Eigenvalue and Eigenvector Problems An attempt to introduce eigenproblems Radu Trîmbiţaş Babeş-Bolyai University April 8, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) Eigenvalue and Eigenvector Problems

More information

Chapter 2 Notes, Linear Algebra 5e Lay

Chapter 2 Notes, Linear Algebra 5e Lay Contents.1 Operations with Matrices..................................1.1 Addition and Subtraction.............................1. Multiplication by a scalar............................ 3.1.3 Multiplication

More information

Lecture 2 INF-MAT : , LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky

Lecture 2 INF-MAT : , LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky Lecture 2 INF-MAT 4350 2009: 7.1-7.6, LU, symmetric LU, Positve (semi)definite, Cholesky, Semi-Cholesky Tom Lyche and Michael Floater Centre of Mathematics for Applications, Department of Informatics,

More information

Gaussian Elimination and Back Substitution

Gaussian Elimination and Back Substitution Jim Lambers MAT 610 Summer Session 2009-10 Lecture 4 Notes These notes correspond to Sections 31 and 32 in the text Gaussian Elimination and Back Substitution The basic idea behind methods for solving

More information

Linear Algebra Formulas. Ben Lee

Linear Algebra Formulas. Ben Lee Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries

More information

Scientific Computing: Solving Linear Systems

Scientific Computing: Solving Linear Systems Scientific Computing: Solving Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 September 17th and 24th, 2015 A. Donev (Courant

More information

Mathematical Foundations

Mathematical Foundations Chapter 1 Mathematical Foundations 1.1 Big-O Notations In the description of algorithmic complexity, we often have to use the order notations, often in terms of big O and small o. Loosely speaking, for

More information

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation

More information

33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM

33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM 33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM (UPDATED MARCH 17, 2018) The final exam will be cumulative, with a bit more weight on more recent material. This outline covers the what we ve done since the

More information