Theorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form. k k n-k. m-k k m-k n-k

Size: px
Start display at page:

Download "Theorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form. k k n-k. m-k k m-k n-k"

Transcription

1 I. REVIEW OF LINEAR ALGEBRA A. Equivalence Definition A1. If A and B are two m x n matrices, then A is equivalent to B if we can obtain B from A by a finite sequence of elementary row or elementary column operations. Theorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form I O O O k k n-k m-k k m-k n-k Theorem A.2. Two m x n matrices A and B are equivalent if and only if B = PAQ for some nonsingular matrices P and Q. Theorem A.3. An n x n matrix A is nonsingular if and only if A is equivalent to I n. B. Rank Definition B1. Let +, * a 11 a a 1n * * * A = * a 21 a a 2n * *... * *... * * a m1 a a mn *. - be an m x n matrix. The rows of A, considered as vectors in R n, span a subspace of R n, called the row space of A. Similarly, the columns of A,considered as vectors in R m, span a subspace of R m called the column space of A. Definition B2. The dimension of the row(column)space of A is called the row(column) rank of A. 1

2 Theorem B.1. The row rank and column rank of the matrix A = [a ij ] are equal. Since the row and column ranks of a matrix are equal, we shall now merely refer to the rank of a matrix. Note that rank I n =n. Theorem A.2 states that A is equivalent to B if and only if there exist nonsingular matrices P and Q such that B = PAQ. If A is equivalent to B, then rank A = rank B, for rank B = rank(paq) = rank(pa) = rank A. We also recall from the previous section 7 that if A is an m by n matrix, then A is equivalent to a matrix, +, * I k 0 * C = * * * 0 0 *. - Now rank A = rank C = k. C. Determinants The minor *M ij * of the square (n x n) matrix A is the determinant of (n-1)x(n-1) matrix M ij formed from A minus the ith row and jth column. The cofactor of element a ij is C ij ' (&1) i%j *M ij * Laplace expansion of determinant *A* ' j n i ' 1 a ij C ij for constant j or *A* ' j n j ' 1 a ij C ij for constant i. Theorem C.1. If A is a matrix, then *A* = *A T *. 2

3 Theorem C.2. If matrix B results from matrix A by interchanging two rows(columns) of A, then *B* = -*A*. Theorem C.3. If two rows(columns) of A are equal, then *A* = 0. Theorem C.4. If a row(column) of A consists entirely of zeros, then *A* = 0. Theorem C.5. If B is obtained from A by multiplying a row(column) of A by a real number c, then *B* = c*a*. Theorem C.6. If B = [b ij ] is obtained from A = [a ij ] by adding to each element of the rth row(column) c times the corresponding element of the sth row (column)(r û then *B* = *A*. Theorem C.7. If a matrix A = [a ij ] is upper (lower) triangular, then *A* = a 11 a nn ; that is, the determinant of a triangular matrix is the product of the elements on main diagonal. the s), Theorem C.8. If A is an n x n matrix, then A is nonsingular if and only if *A* û 0. Theorem C.9. If A and B are n x n matrices, then *AB* = *A*@*B*. D. Special Matrices (1) The adjoint of A is Adj(A) = [C ji ] (2) Inverse of A; A -1 = Adj(A)/*A* (3) Symmetric matrix; A = A T (4) Skew-symmetric matrix; A = -A T for square, real A. (5) Associate matrix of A; (A * ) T (6) Hermitian matrix; A equals its associate (7) Involutary matrix; AA = I (8) Orthogonal Matrix; A -1 = A T 3

4 (9) Toeplitz matrix; a(i,j)=r(i-j) (10) Autocorrelation matrix; A = E[x@x T ] for n by 1 vector x. A is Toeplitz if x represents evenly spaced samples from a wide-sense stationary random process. (11) Autocovariance matrix; A = E[(x-E[x])(x-E[x]) T ] for n by 1 vector x. A is Toeplitz if x represents evenly spaced samples from a wide-sense stationary random process. E. Positive Definite Matrices An n x n matrix C with the property that x T Cx > 0 for any nonzero vector x in R n is called positive definite. Such a matrix is nonsingular, for if C is singular, then the homogeneous system Cx = 0 has a nontrivial solution x o. Then x o T Cx o = 0, contradicting the requirement that x T Cx > 0 for nonzero x. Conversely, if C = [c ij ] is any n x n symmetric matrix that is positive definite (that is, x T Cx > 0 if x is a nonzero vector in R n ), then we define (",ß),for " = a 1 " 1 + a 2 " 2 + a n " n and ß = b 1 " 1 + b 2 " 2 + b n " n (, ) ' j n n j i ' 1 j ' 1 a i c ij b j. in V by It is not difficult to show that this defines an inner product on V. Theorem E.1. A real, symmetric matrix C is positive definite iff there exists a nonsingular matrix A such that C=A T A Theorem E.2. A T A is positive definite for nonsingular A since x T (A T A)x = (Ax) T (Ax) = 5Ax5 2 2 > 0 for every nonzero n by 1 vector x. Theorem E.3. A matrix C is positive definite (semidefinite) iff its eigenvalues are all positive (nonegative). Similarly, C is negative definite (semidefinite) iff its eigenvalues are all negative. C is indefinite if it has both positive and negative eigenvalues. Theorem E.4. A matrix C is positive definite iff every upper left hand determinant (leading principal minor) of C is positive. This is called Sylvester's test. 4

5 Theorem E.5. The Toeplitz matrix formed from an autocorrelation function is at least positive semidefinite. Theorem E.6. The non-toeplitz autocorrelation matrix A = E[x@x T ] is also positive semidefinite. F. Orthogonal Matrices Definition F1. A square matrix A is called orthogonal if A -1 = A T. Of course, we can also say that A is orthogonal if A T A = I n. Theorem F.1 All the roots of the characteristic polynomial of a real symmetric matrix are real numbers. Theorem F.2 If A is a symmetric n x n matrix, then the eigenvectors that belong to distinct eigenvalues of A are orthogonal. Theorem F.3. The n x n matrix A is orthogonal if and only if the columns(and rows) of A form an orthonormal set. Theorem F.4 If A is a symmetric n x m matrix, then there exists an orthogonal matrix P such that P -1 AP = P T AP = D, a diagonal matrix. The eigenvalues of A lie on the main diagonal of D. G. Characteristic Equation, Eigenvalues, And Eigenvectors The characteristic equation for an n x n matrix A is P(8) = *8I-A* = 0, or P(8) = 8 n +a 1 8 n-1 +a 2 8 n a n-1 8+a n = 0 or P(8) = (8-8 1 )(8-8 2 )...(8-8 n ) Now 5

6 P(0) = *-A* = (-1) n *A* and P(0) = (-1) n ( n ) so *A* = n For each eigenvalue 8 i, the exists an eigenvector x i which is a solution to [8 i I-A]x i = 0 Also, each x i can be found as any nonzero column of Adj[8 i I-A]. The modal matrix is one formed from columns, k i x i. Theorem G.1. The eigenvalues of a symmetric matrix A are real. Theorem G.2. The eigenvectors, corresponding to distinct eigenvalues of a symmetric matrix A, form an orthogonal set (Same as Thm. F.2.). Theorem G.3. If A has eigenvalues 8 i, then the matrix A k has eigenvalues 8 i k. This can be proved using Thm. F.4. Theorem G.4. If the nonsingular matrix A has eigenvalues 8 i, then the matrix A -1 has eigenvalues 1/8 i. Theorem G.5. If A has eigenvalues 8 i, then the matrix I@k + A has eigenvalues 8 i +k. Theorem G.6. Two matrices are similar if they have the same eigenvalues. If P is a nonsingular matrix and A = P@S@P -1 then A and S are similar, since if 8 is an eigenvalue of S, then S@x = 8@x, P@S@P -1 (P@x) = 8@(P@x) 6

7 H. Spectral Decomposition Expanding on Thm. F.4, any arbitrary complex-valued n by n matrix A with n linearly independent eigenvectors can be decomposed as where A = P@S@P -1 S = diag(8 1,...,8 n ), P = [p 1,...,p n ], 8 i is the ith eigenvalue of A and p i is the corresponding eigenvector. This is proved as follows. By the definition of eigenvectors and eigenvalues, AP = PS, from which we get Ap i = 8 i p i so A = P@S@P -1 For the general case, the eigenvalues can be positive or negative. If A is symmetric, then S is real. If A is positive semidefinite, then the diagonal elements of S (eigenvalues of A) are nonnegative (from Thm. E.3). 7

8 II. The Singular Value Decomposition (SVD) A. Orthogonality And The SVD The singular value decomposition-the SVD-is a powerful computational tool for analyzing matrices and problems involving matrices which has applications in many fields. In the remaining sections, we shall define the SVD, describe some other applications, and present an algorithm for computing it. The algorithm is representative of algorithms currently used for various matrix eigenvalue problems and serves as an introduction to computational techniques for these problems as well. Although it is still not widely known, the singular value decomposition has a fairly long history. Much of the fundamental work was done by Gene Golub and his colleagues W. Kahan, Peter Businger, and Christian Reinsch. Our discusssion will be based largely on a paper by Golub and Reinsch(1971). The underlying matrix eigenvalue algorithms have been developed by J.G.F. Francis, H. Rutishauser, and J.H. Wilkinson and are presented in Wilkinson s book(1965). Recent books by Lawson and Hanson(1974) and Stewart(1973)discuss the SVD as well as many related topics. In elementary linear algebra, a set of vectors is defined to be independent if none of them can be expressed as a linear combination of the others. In computational linear algebra, it is very useful to have a quantative notion of the "amount" of independence. We would like to define a quantity that reflects the fact that, for example, (1, 0, 0), (0, 1, 0), and (0, 0, 1) are very independent, whereas (1.01, 1.00, 1.00), (1.00, 1.01, 1.00), and (1.00, 1.00, 1.01) are almost dependent. Since two vectors are dependent if they are parallel, it is reasonable to regard them as very independent if they are perpendicular or orthogonal. Using a superscript T to denote the transpose of a vector or matrix, two vectors u and v are orthogonal if their inner product is zero, that is, if Moreover, a vector u has length 1 if u T v = 0. u T u = 1. A square matrix is called orthogonal (see Def. F1) if its columns are mutually orthogonal vectors each of length 1. Thus a matrix U is orthogonal if U T U = I, the identity matrix. 8

9 Note that an orthogonal matrix is automatically nonsingular, since U -1 = U T. In fact, we shall soon make precise the idea that an orthogonal matrix is very nonsingular and that its columns are very independent. The simplest examples of orthogonal matrices are planar rotations of the form +, * cos2 sin2 * U = * * *-sin2 cos2 *. - If x is a vector in 2-space, then Ux is the same vector rotated through an angle 2. It is useful to associate orthogonal matrices with such rotations, even though in higher dimensions orthogonal matrices can be more complicated. For example, +, * * 1 * * U = ))) * * 49 * * * *. - is orthogonal but cannot be interpreted as a simple plane rotation. Multiplication by orthogonal matrices does not change such important geometrical quantities as the length of a vector or the angle between two vectors. Orthogonal matrices also have highly desirable computational properties because they do not magnify errors. For any matrix A and any two orthogonal matrices U and V, consider the matrix G defined by G = U T AV. If u j and v j are the columns of U and V, respectively, then the individual components of G are F ij = u i T Av j. The idea behind the singular value decomposition is that by proper choice of U and V it is possible to make most of the F ij zero; in fact, it is possible to make G diagonal with nonnegative diagonal entries. Consequently, we make the following definition. A singular value decomposition of an m-by-n real matrix A is any factorization of the form A = UGV T, where U is an m-by-m orthogonal matrix, V is an n-by-n orthogonal matrix, and G is an m-by-n 9

10 diagonal matrix with F ij = 0 if iûj and F ii = F i $ 0. The quantities F i are called the singular values of A, and the columns of U and V are called the left and right singular vectors. Note the similarity to Thm. F.4 and section I.H. Readers familiar with matrix eigenvalues should note that the matrices AA T and A T A have the same nonzero eigenvalues and that the singular values of A are the positive square roots of these eigenvalues. Moreover, the left and right singular vectors are particular choices of the eigenvectors of AA T and A T A, respectively. (See theorems E.1, E.2, and E.3). In the language of abstract linear algebra, the matrix A is the representation of some linear transformation in a particular coordinate system. By making one orthogonal change of coordinates in the domain of this transformation and a second orthogonal change of coordinates in the range, the representation becomes diagonal. B. Rank And Condition Number The notion of the rank of a matrix is fundamental to much of linear algebra (see section I.B.). The usual definition is the maximum number of independent columns, or, equivalently, the order of the maximal nonzero subdeterminant in the matrix. Using such a definition, it is difficult to actually determine the rank of a general matrix in practice. However, if the matrix is diagonal, it is clear that its rank is the number of nonzero diagonal entries. If a set of independent vectors is multiplied by an orthogonal matrix, the resulting set is still independent. In other words, the rank of a general matrix A is equal to the rank of the diagonal matrix G in its SVD. Consequently, a practical definition of the rank of a matrix is the number of nonzero singular values. We shall use the letter k to denote rank. An m-by-n matrix with m $ n is said to be of full rank if k = n or rank deficient if k < n. For square matrices, the more common terms nonsingular and singular are often used for full rank and rank deficient, respectively. Since the rank of a matrix must always be an integer, it is necessarily a discontinuous function of the elements of the matrix. Arbitrarily small changes(such as roundoff errors) in a rank-deficient matrix can make all of its singular values nonzero and hence create a matrix which is technically of full rank. In practice, we work with the effective rank, the number of singular values greater than some prescribed tolerance which reflects the accuracy of the data. This is also a discontinuous function, but the discontinuities are much less numerous and troublesome than those of the theoretical rank. The great advantage of the use of the SVD in determining the rank of a matrix is that decisions need be made only about the negligibility of single numbers-the small singular 10

11 values-rather than vectors or sets of vectors. We can now precisely define the measure of independence mentioned earlier. The condititon number of a matrix A of full rank is cond(a) = F max /F min where F max and F min are the largest and smallest singular values of A. If A is rank deficient, then F min = 0 and cond(a) is said to be infinite. Clearly, cond(a) $ 1. If cond(a) is close to 1, then the columns of A are very independent. If cond(a) is large, then the columns of A are nearly dependent. If A is square, then terms like nearly singular or far from singular can be given fairly precise meanings. A matrix A is considered to be more singular than a matrix B if cond(a) > cond(b). If A is orthogonal, then cond(a) = 1, and so the columns of an orthogonal matrix are as independent as possible. Conversely, if cond(a) = 1, then it turns out that A must be a scalar multiple of an orthogonal matrix. Two common norms for the m by n matrix A are 2A2 ' ( j m n j i ' 1 j ' 1 *a(i,j)* 2 ).5 and 2A2 ' max 2Ax2 2 2x2 2 ' max 2x2 2 '1 2Ax2 2 which are respectively the Euclidean matrix norm or Frobenius norm, and the spectral norm. Using the singular values F i where F 1 is assumed to be the largest, the two norms are expressed respectively as 2A2 ' ( j n i ' 1 2 i ).5, 5A5 = F 1 11

12 C. Evaluating Determinants and Finding Singular Matrices Let A(z) be a square matrix which depends in a possibly complicated way on some parameter z. Consider finding a value, or values, of z for which A(z) is singular. Equivalently, find z for which the determinant of A(z) is zero. Using det to denote determinant, det(a) = det(u)@det(g)@det(v T ). The determinant of an orthogonal matrix is ±1, and the determinant of a diagonal matrix is simply the product of its elements, so det(a) = ±F n. Computing determinants can be tricky because the value can vary over a huge range, and floating-point underflows and oveflows are frequent. Many problems which are formulated in terms of determinants do not require the actual value of the determinant but simply some indication of when it is zero. It is theoretically possible to compute *det(a)* by taking the product of the singular values of A. But in most situations, it is sufficient to use only the smallest singular value and thereby avoid underflow/overflow and other numerical difficulties. D. The Eigenvalue Problem This is not really a practical application of SVD but rather an indication of how the SVD is related to a larger class of important matrix problems. Let A be a square matrix. The eigenvalues of A are the numbers 8 for which Ax = 8x has a nonzero solution x. Since this is equivalent to requiring det(a-8i) = 0 the eigenvalues could theoretically be computed by finding the roots of this polynomial. However, consider 12

13 +, * 1 e * A = * * * e 1 *. - for some small but not negligible number e. The true eigenvalues are 8 1 = 1 + e and 8 2 = 1 - e. The polynomial det(a-8i) is (1-e 2 ). If e is small, then e 2 is much smaller, and it is necessary to have twice the precision in the coefficients of the polynomial than is present in either the matrix elements or the eigenvalues. This difficulty becomes even more pronounced for higher-order matrices. Consequently, modern methods for computing eigenvalues avoid the use of polynomials or polynomial root finders. The connection between SVD and eigenvalues is simplest for matrices which are symmetric and positive semidefinite. Symmetry is easy to define and check-a is symmetric if a ij = a ji for all i and j. Positive semidefiniteness is much more elusive-positive semidefinite means that x T Ax $ 0 for all x. Roughly this means that the diagonal elements of A are fairly large compared to the off-diagonal elements. In fact, a sufficient condition is that for each i a ii $ j j û 1 *a ij * but this is not necessary (see section I.E.). It is not difficult to show that if A is real and symmetric, then all its eigenvalues are real, and if A is positive semidefinite, then all its eigenvalues are nonnegative. If A = UGV T and A = A T, then A 2 = A T A = VG T U T UGV T = VG 2 V T Thus, A 2 V = VG 2. Letting v j denote the columns of V and looking at the jth column of this equation, we find A 2 v j = F 2 jv j Since the eigenvalues of A 2 are the squares of the eigenvalues of A (from Thm. G.3) and since the 13

14 eigenvalues of A are nonnegative, we conclude that the eigenvalues of a symmetric, positivesemidefinite matrix (see Thm. E.3) are equal to its singular values and that the eigenvectors are the columns of V. Thus, V is a modal matrix for A. If A is symmetric but not positive semidefinite, then some of its eigenvalues are negative. It turns out that the absolute values of the eigenvalues are equal to the singular values and that the signs of the eigenvalues can be recovered by comparing the columns of U with those of V, but we shall not go into details. If A is not symmetric, there is no simple connection between its eigenvalues and its singular values. This shows that the SVD algorithm could be used to compute the eigenvalues of symmetric matrices. However, this is not particularly efficient and not recommended. Our only point is that the SVD is closely related to matrix eigenvalue problems. Careful study of the SVD algorithm will help in understanding eigenvalue algorithms as well. E. Linear Sets of Equations Let A be a given m by n matrix, where m$n, and let b be a given m vector. We want to find all n vectors x which solve A@x = b (1) Note that this includes the cases where A is square and singular, and square and nonsingular. Important questions include; (1) Are the equations consistent? (2) Do any solutions exist? (3) Is the solution unique? (4) Do any nonzero solutions exist? (5) What is the general form of the solution? There are many methods for answering these questions, but the SVD is the only reliable method. Using the SVD of A, equation (1) becomes UGV T x = b (2) 14

15 and hence Gz = d (3) where z = V T x and d = U T b. The system of equations in (3) is diagonal and is therefore easily studied. It breaks up into as many as three sets, depending on the values of the dimensions m and n and the rank k, the number of nonzero singular values. The three sets are: F j z j = d j, if j # n and F j û 0, (4) 0@z j = d j, if j # n and F j = 0, (5) 0 = d j, if j > n. (6) Theorem E.1. The equations in (1) are consistent and a solution exists iff d j =0 whenever F j =0 or j>n. Theorem E.2. If k<n, then the z j associated with a zero F j can be given an arbitrary value and still yield a solution to the system. When transformed back to the original coordinates by x = V@z, (7) these arbitrary components of z serve to parameterize the space of all possible solutions x. The Linear Least-Squares Problem This is an extension of the previous problem, but we now seek n vectors x for which Ax is only approximately equal to b in the sense that the length of the residual vector, r = A@x - b, is minimized. The problem is to pick an x which minimizes 2r2 2 ' j m i ' 1 r 2 i (8) 15

16 If A has full rank, then the solution to x is unique and may be solved more efficiently by methods other than the SVD, such as the Toeplitz recursion. However, the SVD also handles the rank deficient case. Since orthogonal matrices preserve the norm (see section II.A.), equation (8) can be rewritten as 5r5 = 5U T (AVV T x - b)5 = 5Gz - d5 (9) using the SVD and the fact that minimizing the norm squared of r is equivalent to minimizing the norm of r. The vector z which minimizes 5r5 is given by z j = d j /F j, if F j û 0, z j = anything, if F j = 0. Therefore, if the problem is rank deficient, the solution which minimizes 5r5 is not unique. In this rank deficient case, it is often desirable to pick the minimum norm solution for z and therefore for x. This gives us a unique solution. This solution is obtained by setting z j = 0, if F j = 0 Once z is determined for the full rank or deficient rank cases, we find the solution vector x using equation (7). The error 5r5 2 is calculated in either case as 5r5 2 = G d j 2 10) for values of j where F j = 0. 16

17 F. The Discrete Karhunen-Loeve Transform (KLT) In many statistical pattern recognition problems and maximum likelihood problems we are given a random vector x, of dimension n, which has a joint Gaussian density with parameters E[x] = m x, C x = E[(x - m x )(x - m x ) T )] m x and C x are the mean vector and autocovariance matrix of x. C x is Toeplitz if x represents evenly spaced samples from a stationary random process. (See (11) in section I.D.) We will assume that C x is positive definite. The discrete Karhunen-Loeve Transform (KLT) is a transform, z = A@x (1) such that; (1) The covariance matrix C z = E[(z - m z )(z - m z ) T ] is diagonal so that A is an orthogonal matrix, and the elements z(i) of z are independent Gaussian random variables. (2) The sequence F ii = C z (i,i) is non-increasing. (3) No other transform satisfies (2) with larger F ii values. Define the SVD of C x as C x = U@G@V T. (2) Using equation (1) and x = A -1 z (3) we get 17

18 C x = E[(A -1 z - A -1 m z )(A -1 z - A -1 m z ) T ] = E[A - m z )(z - m z ) T A] = A (4) Since A is an orthogonal matrix and since C z (i,i) are nonegative, equation (4) represents an SVD of C x. Therefore, for the KLT, we take the SVD of C x as in equation (2) and get V = U, A = U T = V T, C z = G, m z = U T x Thus the transformation matrix A or U T diagonalizes the covariance matrix of z. The rows of A form an orthonormal set of n basis vectors. G. Summary (1) A singular value decomposition of an m-by-n real matrix A is any factorization of the form A = UGV T, where U is an m-by-m orthogonal matrix, V is an n-by-n orthogonal matrix, and G is an m-by-n diagonal matrix with F ij = 0 if iûj and F ii = F i $ 0. The quantities F i are called the singular values of A, and the columns of U and V are called the left and right singular vectors. (2) The rank of A is the number of nonzero singular values of A. (3) The condititon number of a matrix A of full rank is cond(a) = F max /F min, where F max and F min are the largest and smallest singular values of A. 18

19 (4) det(a) = ±F n. (5) The columns of U and V are respectively eigenvectors of AA T and A T A. These sets of columns are called the left singular vectors (for U) and the right singular vectors (for V). (6) The singular values are the positive square roots of the eigenvalues of A T A. (7) If A is symmetric and positive semidefinite, then the singular values of A are the eigenvalues of A, and the SVD is a spectral decomposition of A. (See section I part H.) If A is an autocorrelation matrix, it satisfies this condition. (See section I part E.) (8) The SVD can be used to solve linear sets of equations, including those involving least-squares, even when the A matrix is rank deficient. (9) The orthogonal KLT transformation matrix for a random vector x with covariance matrix C x = U@G@V T is defined as A = U T The diagonal covariance matrix for the transformed vector z is C z = G 19

EE731 Lecture Notes: Matrix Computations for Signal Processing

EE731 Lecture Notes: Matrix Computations for Signal Processing EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten

More information

Chapter 3 Transformations

Chapter 3 Transformations Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases

More information

Fundamentals of Engineering Analysis (650163)

Fundamentals of Engineering Analysis (650163) Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det

1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix

More information

ELE/MCE 503 Linear Algebra Facts Fall 2018

ELE/MCE 503 Linear Algebra Facts Fall 2018 ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2

More information

Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in

Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column

More information

MAT Linear Algebra Collection of sample exams

MAT Linear Algebra Collection of sample exams MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

More information

Linear Algebra: Matrix Eigenvalue Problems

Linear Algebra: Matrix Eigenvalue Problems CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given

More information

Review problems for MA 54, Fall 2004.

Review problems for MA 54, Fall 2004. Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on

More information

c c c c c c c c c c a 3x3 matrix C= has a determinant determined by

c c c c c c c c c c a 3x3 matrix C= has a determinant determined by Linear Algebra Determinants and Eigenvalues Introduction: Many important geometric and algebraic properties of square matrices are associated with a single real number revealed by what s known as the determinant.

More information

Linear Systems and Matrices

Linear Systems and Matrices Department of Mathematics The Chinese University of Hong Kong 1 System of m linear equations in n unknowns (linear system) a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.......

More information

Matrices A brief introduction

Matrices A brief introduction Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 41 Definitions Definition A matrix is a set of N real or complex

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

Linear Algebra Review. Vectors

Linear Algebra Review. Vectors Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors

More information

Linear Algebra: Lecture notes from Kolman and Hill 9th edition.

Linear Algebra: Lecture notes from Kolman and Hill 9th edition. Linear Algebra: Lecture notes from Kolman and Hill 9th edition Taylan Şengül March 20, 2019 Please let me know of any mistakes in these notes Contents Week 1 1 11 Systems of Linear Equations 1 12 Matrices

More information

MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, Chapter 1: Linear Equations and Matrices MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

More information

22.3. Repeated Eigenvalues and Symmetric Matrices. Introduction. Prerequisites. Learning Outcomes

22.3. Repeated Eigenvalues and Symmetric Matrices. Introduction. Prerequisites. Learning Outcomes Repeated Eigenvalues and Symmetric Matrices. Introduction In this Section we further develop the theory of eigenvalues and eigenvectors in two distinct directions. Firstly we look at matrices where one

More information

Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

More information

Lecture 7. Econ August 18

Lecture 7. Econ August 18 Lecture 7 Econ 2001 2015 August 18 Lecture 7 Outline First, the theorem of the maximum, an amazing result about continuity in optimization problems. Then, we start linear algebra, mostly looking at familiar

More information

Math Linear Algebra Final Exam Review Sheet

Math Linear Algebra Final Exam Review Sheet Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

More information

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0. Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the

More information

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88 Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

More information

3 (Maths) Linear Algebra

3 (Maths) Linear Algebra 3 (Maths) Linear Algebra References: Simon and Blume, chapters 6 to 11, 16 and 23; Pemberton and Rau, chapters 11 to 13 and 25; Sundaram, sections 1.3 and 1.5. The methods and concepts of linear algebra

More information

NOTES FOR LINEAR ALGEBRA 133

NOTES FOR LINEAR ALGEBRA 133 NOTES FOR LINEAR ALGEBRA 33 William J Anderson McGill University These are not official notes for Math 33 identical to the notes projected in class They are intended for Anderson s section 4, and are 2

More information

Lecture Summaries for Linear Algebra M51A

Lecture Summaries for Linear Algebra M51A These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture

More information

a 11 a 12 a 11 a 12 a 13 a 21 a 22 a 23 . a 31 a 32 a 33 a 12 a 21 a 23 a 31 a = = = = 12

a 11 a 12 a 11 a 12 a 13 a 21 a 22 a 23 . a 31 a 32 a 33 a 12 a 21 a 23 a 31 a = = = = 12 24 8 Matrices Determinant of 2 2 matrix Given a 2 2 matrix [ ] a a A = 2 a 2 a 22 the real number a a 22 a 2 a 2 is determinant and denoted by det(a) = a a 2 a 2 a 22 Example 8 Find determinant of 2 2

More information

Econ Slides from Lecture 7

Econ Slides from Lecture 7 Econ 205 Sobel Econ 205 - Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for

More information

Chapter Two Elements of Linear Algebra

Chapter Two Elements of Linear Algebra Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to

More information

Foundations of Matrix Analysis

Foundations of Matrix Analysis 1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the

More information

Linear Algebra Highlights

Linear Algebra Highlights Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to

More information

7. Symmetric Matrices and Quadratic Forms

7. Symmetric Matrices and Quadratic Forms Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value

More information

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the

More information

Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.

Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same. Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm

University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm Name: The proctor will let you read the following conditions

More information

3 Matrix Algebra. 3.1 Operations on matrices

3 Matrix Algebra. 3.1 Operations on matrices 3 Matrix Algebra A matrix is a rectangular array of numbers; it is of size m n if it has m rows and n columns. A 1 n matrix is a row vector; an m 1 matrix is a column vector. For example: 1 5 3 5 3 5 8

More information

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1

More information

Linear Least-Squares Data Fitting

Linear Least-Squares Data Fitting CHAPTER 6 Linear Least-Squares Data Fitting 61 Introduction Recall that in chapter 3 we were discussing linear systems of equations, written in shorthand in the form Ax = b In chapter 3, we just considered

More information

Quantum Computing Lecture 2. Review of Linear Algebra

Quantum Computing Lecture 2. Review of Linear Algebra Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces

More information

Chapter 2:Determinants. Section 2.1: Determinants by cofactor expansion

Chapter 2:Determinants. Section 2.1: Determinants by cofactor expansion Chapter 2:Determinants Section 2.1: Determinants by cofactor expansion [ ] a b Recall: The 2 2 matrix is invertible if ad bc 0. The c d ([ ]) a b function f = ad bc is called the determinant and it associates

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

Calculation in the special cases n = 2 and n = 3:

Calculation in the special cases n = 2 and n = 3: 9. The determinant The determinant is a function (with real numbers as values) which is defined for quadratic matrices. It allows to make conclusions about the rank and appears in diverse theorems and

More information

1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true?

1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true? . Let m and n be two natural numbers such that m > n. Which of the following is/are true? (i) A linear system of m equations in n variables is always consistent. (ii) A linear system of n equations in

More information

(1) for all (2) for all and all

(1) for all (2) for all and all 8. Linear mappings and matrices A mapping f from IR n to IR m is called linear if it fulfills the following two properties: (1) for all (2) for all and all Mappings of this sort appear frequently in the

More information

9. The determinant. Notation: Also: A matrix, det(a) = A IR determinant of A. Calculation in the special cases n = 2 and n = 3:

9. The determinant. Notation: Also: A matrix, det(a) = A IR determinant of A. Calculation in the special cases n = 2 and n = 3: 9. The determinant The determinant is a function (with real numbers as values) which is defined for square matrices. It allows to make conclusions about the rank and appears in diverse theorems and formulas.

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2 MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

More information

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian

More information

A Review of Linear Algebra

A Review of Linear Algebra A Review of Linear Algebra Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab: Implementations

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra

SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra 1.1. Introduction SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear algebra is a specific branch of mathematics dealing with the study of vectors, vector spaces with functions that

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

1. General Vector Spaces

1. General Vector Spaces 1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

More information

MAT 2037 LINEAR ALGEBRA I web:

MAT 2037 LINEAR ALGEBRA I web: MAT 237 LINEAR ALGEBRA I 2625 Dokuz Eylül University, Faculty of Science, Department of Mathematics web: Instructor: Engin Mermut http://kisideuedutr/enginmermut/ HOMEWORK 2 MATRIX ALGEBRA Textbook: Linear

More information

1 Matrices and Systems of Linear Equations. a 1n a 2n

1 Matrices and Systems of Linear Equations. a 1n a 2n March 31, 2013 16-1 16. Systems of Linear Equations 1 Matrices and Systems of Linear Equations An m n matrix is an array A = (a ij ) of the form a 11 a 21 a m1 a 1n a 2n... a mn where each a ij is a real

More information

Chapter 3. Determinants and Eigenvalues

Chapter 3. Determinants and Eigenvalues Chapter 3. Determinants and Eigenvalues 3.1. Determinants With each square matrix we can associate a real number called the determinant of the matrix. Determinants have important applications to the theory

More information

14 Singular Value Decomposition

14 Singular Value Decomposition 14 Singular Value Decomposition For any high-dimensional data analysis, one s first thought should often be: can I use an SVD? The singular value decomposition is an invaluable analysis tool for dealing

More information

Numerical Linear Algebra Homework Assignment - Week 2

Numerical Linear Algebra Homework Assignment - Week 2 Numerical Linear Algebra Homework Assignment - Week 2 Đoàn Trần Nguyên Tùng Student ID: 1411352 8th October 2016 Exercise 2.1: Show that if a matrix A is both triangular and unitary, then it is diagonal.

More information

Repeated Eigenvalues and Symmetric Matrices

Repeated Eigenvalues and Symmetric Matrices Repeated Eigenvalues and Symmetric Matrices. Introduction In this Section we further develop the theory of eigenvalues and eigenvectors in two distinct directions. Firstly we look at matrices where one

More information

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

Applied Linear Algebra

Applied Linear Algebra Applied Linear Algebra Gábor P. Nagy and Viktor Vígh University of Szeged Bolyai Institute Winter 2014 1 / 262 Table of contents I 1 Introduction, review Complex numbers Vectors and matrices Determinants

More information

Lecture 1: Review of linear algebra

Lecture 1: Review of linear algebra Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, least-squares and least-norm solutions Subspaces, basis, and dimension Change of basis and similarity transformations

More information

SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra

SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to 1.1. Introduction Linear algebra is a specific branch of mathematics dealing with the study of vectors, vector spaces with functions that

More information

Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =

Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = 30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can

More information

Chapter 7. Linear Algebra: Matrices, Vectors,

Chapter 7. Linear Algebra: Matrices, Vectors, Chapter 7. Linear Algebra: Matrices, Vectors, Determinants. Linear Systems Linear algebra includes the theory and application of linear systems of equations, linear transformations, and eigenvalue problems.

More information

Image Registration Lecture 2: Vectors and Matrices

Image Registration Lecture 2: Vectors and Matrices Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this

More information

The Singular Value Decomposition

The Singular Value Decomposition The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will

More information

Review of some mathematical tools

Review of some mathematical tools MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical

More information

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017 Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...

More information

Introduction to Matrix Algebra

Introduction to Matrix Algebra Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

More information

Mathematics. EC / EE / IN / ME / CE. for

Mathematics.   EC / EE / IN / ME / CE. for Mathematics for EC / EE / IN / ME / CE By www.thegateacademy.com Syllabus Syllabus for Mathematics Linear Algebra: Matrix Algebra, Systems of Linear Equations, Eigenvalues and Eigenvectors. Probability

More information

Linear Algebra in Actuarial Science: Slides to the lecture

Linear Algebra in Actuarial Science: Slides to the lecture Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

8. Diagonalization.

8. Diagonalization. 8. Diagonalization 8.1. Matrix Representations of Linear Transformations Matrix of A Linear Operator with Respect to A Basis We know that every linear transformation T: R n R m has an associated standard

More information

Matrix Algebra: Summary

Matrix Algebra: Summary May, 27 Appendix E Matrix Algebra: Summary ontents E. Vectors and Matrtices.......................... 2 E.. Notation.................................. 2 E..2 Special Types of Vectors.........................

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,

More information

15 Singular Value Decomposition

15 Singular Value Decomposition 15 Singular Value Decomposition For any high-dimensional data analysis, one s first thought should often be: can I use an SVD? The singular value decomposition is an invaluable analysis tool for dealing

More information

Cheat Sheet for MATH461

Cheat Sheet for MATH461 Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

Conceptual Questions for Review

Conceptual Questions for Review Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.

More information

MAT 610: Numerical Linear Algebra. James V. Lambers

MAT 610: Numerical Linear Algebra. James V. Lambers MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic

More information

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2017 LECTURE 5

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2017 LECTURE 5 STAT 39: MATHEMATICAL COMPUTATIONS I FALL 17 LECTURE 5 1 existence of svd Theorem 1 (Existence of SVD) Every matrix has a singular value decomposition (condensed version) Proof Let A C m n and for simplicity

More information

Lecture 10: Determinants and Cramer s Rule

Lecture 10: Determinants and Cramer s Rule Lecture 0: Determinants and Cramer s Rule The determinant and its applications. Definition The determinant of a square matrix A, denoted by det(a) or A, is a real number, which is defined as follows. -by-

More information

1 Matrices and Systems of Linear Equations

1 Matrices and Systems of Linear Equations March 3, 203 6-6. Systems of Linear Equations Matrices and Systems of Linear Equations An m n matrix is an array A = a ij of the form a a n a 2 a 2n... a m a mn where each a ij is a real or complex number.

More information

Linear Algebra Review

Linear Algebra Review Chapter 1 Linear Algebra Review It is assumed that you have had a course in linear algebra, and are familiar with matrix multiplication, eigenvectors, etc. I will review some of these terms here, but quite

More information

Linear Algebra Primer

Linear Algebra Primer Introduction Linear Algebra Primer Daniel S. Stutts, Ph.D. Original Edition: 2/99 Current Edition: 4//4 This primer was written to provide a brief overview of the main concepts and methods in elementary

More information

Mathematical Optimisation, Chpt 2: Linear Equations and inequalities

Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl http://dickinson.website version: 12/02/18 Monday 5th February 2018 Peter J.C. Dickinson

More information

Online Exercises for Linear Algebra XM511

Online Exercises for Linear Algebra XM511 This document lists the online exercises for XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Lecture 02 ( 1.1) Online Exercises for Linear Algebra XM511 1) The matrix [3 2

More information

Matrices A brief introduction

Matrices A brief introduction Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 44 Definitions Definition A matrix is a set of N real or complex

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

Math113: Linear Algebra. Beifang Chen

Math113: Linear Algebra. Beifang Chen Math3: Linear Algebra Beifang Chen Spring 26 Contents Systems of Linear Equations 3 Systems of Linear Equations 3 Linear Systems 3 2 Geometric Interpretation 3 3 Matrices of Linear Systems 4 4 Elementary

More information

MATH2210 Notebook 2 Spring 2018

MATH2210 Notebook 2 Spring 2018 MATH2210 Notebook 2 Spring 2018 prepared by Professor Jenny Baglivo c Copyright 2009 2018 by Jenny A. Baglivo. All Rights Reserved. 2 MATH2210 Notebook 2 3 2.1 Matrices and Their Operations................................

More information

4. Determinants.

4. Determinants. 4. Determinants 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 2 2 determinant 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 3 3 determinant 4.1.

More information

33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM

33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM 33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM (UPDATED MARCH 17, 2018) The final exam will be cumulative, with a bit more weight on more recent material. This outline covers the what we ve done since the

More information

ELEMENTARY LINEAR ALGEBRA

ELEMENTARY LINEAR ALGEBRA ELEMENTARY LINEAR ALGEBRA K R MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND First Printing, 99 Chapter LINEAR EQUATIONS Introduction to linear equations A linear equation in n unknowns x,

More information

Linear Algebra Primer

Linear Algebra Primer Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

More information

Elementary Linear Algebra

Elementary Linear Algebra Matrices J MUSCAT Elementary Linear Algebra Matrices Definition Dr J Muscat 2002 A matrix is a rectangular array of numbers, arranged in rows and columns a a 2 a 3 a n a 2 a 22 a 23 a 2n A = a m a mn We

More information