Jim Lambers MAT 610 Summer Session Lecture 1 Notes


 Tamsin Horton
 1 years ago
 Views:
Transcription
1 Jim Lambers MAT 60 Summer Session Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra by numerical methods that can be implemented on a computer We begin with a brief discussion of the problems that will be discussed in this course, and then establish some fundamental concepts that will serve as the foundation of our discussion of various numerical methods for solving these problems Systems of Linear Equations One of the most fundamental problems in computational mathematics is to solve a system of n linear equations a x + a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2 a m x + a m2 x a mn x n = b m for the n unknowns x, x 2,, x n Many data fitting problems, such as polynomial interpolation or leastsquares approximation, involve the solution of such a system Discretization of partial differential equations often yields systems of linear equations that must be solved Systems of nonlinear equations are typically solved using iterative methods that solve a system of linear equations during each iteration In this course, we will study the solution of this type of problem in detail The Eigenvalue Problem As we will see, a system of linear equations can be described in terms of a single vector equation, in which the product of a known matrix of the coefficients a ij, and an unknown vector containing the unknowns x j, is equated to a known vector containing the righthand side values b i Unlike multiplication of numbers, multiplication of a matrix and a vector is an operation that is difficult to understand intuitively However, given a matrix A, there are certain vectors for which multiplication by A is equivalent to ordinary multiplication This leads to the eigenvalue problem, which consists of finding a nonzero vector x and a number λ such that Ax = λx
2 The number λ is called an eigenvalue of A, and the vector x is called an eigenvector corresponding to λ This is another problem whose solution we will discuss in this course Fundamental Concepts, Algorithms, and Notation Writing a system of equations can be quite tedious Therefore, we instead represent a system of linear equations using a matrix, which is an array of elements, or entries We say that a matrix A is m n if it has m rows and n columns, and we denote the element in row i and column j by a ij We also denote the matrix A by [a ij With this notation, a general system of m equations with n unknowns can be represented using a matrix A that contains the coefficients of the equations, a vectors x that contains the unknowns, and a vector b that contains the quantities on the righthand sides of the equations Specifically, A = a a 2 a n a 2 a 22 a 2n a m a m2 a mn, x = Note that the vectors x and b are represented by column vectors Vector Spaces and Linear Transformations x x 2 x n, b = Matrices are much more than notational conveniences for writing systems of linear equations A matrix A can also be used to represent a linear function f A whose domain and range are both sets of vectors called vector spaces A vector space over a field (such as the field of real or complex numbers) is a set of vectors, together with two operations: addition of vectors, and multiplication of a vector by a scalar from the field Specifically, if u and v are vectors belonging to a vector space V over a field F, then the sum of u and v, denoted by u + v, is a vector in V, and the scalar product of u with a scalar α in F, denoted by αu, is also a vector in V These operations have the following properties: Commutativity: For any vectors u and v in V, u + v = v + u Associativity: For any vectors u, v and w in V, (u + v) + w = u + (v + w) Identity element for vector addition: There is a vector 0, known as the zero vector, such that for any vector u in V, u + 0 = 0 + u = u b b 2 b m 2
3 Additive inverse: For any vector u in V, there is a unique vector u in V such that u + ( u) = u + u = 0 Distributivity over vector addition: For any vectors u and v in V, and scalar α in F, α(u + v) = αu + αv Distributivity over scalar multiplication: For any vector u in V, and scalars α and β in F, (α + β)u = αu + βu Associativity of scalar multiplication: For any vector u in V and any scalars α and β in F, α(βu) = (αβ)u Identity element for scalar multiplication: For any vector u in V, u = u A function f A : V W, whose domain V and range W are vector spaces over a field F, is a linear transformation if it has the properties f A (x + y) = f A (x) + f A (y), f A (αx) = αf A (x), where x and y are vectors in V and α is a scalar from F Subspaces and Bases Before we can explain how matrices can be used to easily describe linear transformations, we must introduce some important concepts related to vector spaces A subspace of a vector space V is a subset of V that is, itself, a vector space In particular, a subset S of V is also a subsapce if it is closed under the operations of vector addition and scalar multiplication That is, if u and v are vectors in S, then the vectors u + v and αu, where α is any scalar, must also be in S In particular, S cannot be a subspace unless it includes the zero vector Often a vector space or subspace can be characterized as the set of all vectors that can be obtained by adding and/or scaling members of a given set of specific vectors For example, R 2 can be described as the set of all vectors that can be obtained by adding and/or scaling the vectors e = [ 0 [ 0, e 2 = 3
4 These vectors comprise what is known as the standard basis of R 2 More generally, given a set of vectors {v, v 2,, v k } from a vector space V, a vector v V is called a linear combination of v, v 2,, v k if there exist constants c, c 2,, c k such that v = c v + c 2 v c k v k = k c i v i i= We then define the span of {v, v 2,, v k }, denoted by span {v, v 2,, v k }, to be the set of all linear combinations of v, v 2,, v k From the definition of a linear combination, it follows that this set is a subspace of V When a subspace is defined as the span of a set of vectors, it is helpful to know whether the set includes any vectors that are, in some sense, redundant, for if this is the case, the description of the subspace can be simplified To that end, we say that a set of vectors {v, v 2,, v k } is linearly independent if the equation c v + c 2 v c k v k = 0 holds if and only if c = c 2 = = c k = 0 Otherwise, we say that the set is linearly dependent If the set is linearly independent, then any vector v in the span of the set is a unique linear combination of members of the set; that is, there is only one way to choose the coefficients of a linear combination that is used to obtain v Given a vector space V, if there exists a set of vectors {v, v 2,, v k } such that V is the span of {v, v 2,, v k }, and {v, v 2,, v k } is linearly independent, then we say that {v, v 2,, v k } is a basis of V Any basis of V must have the same number of elements, k We call this number the dimension of V, which is denoted by dim(v ) Matrix Representation of Linear Transformations If V is a vector space of dimension n over a field, such as R n or C n, and W is a vector space of dimension m, then a linear function f A with domain V and range W can be represented by an m n matrix A whose entries belong to the field Suppose that the set of vectors {v, v 2,, v n } is a basis for V, and the set {w, w 2,, w m } is a basis for W Then, a ij is the scalar by which w i is multiplied when applying the function f A to the vector v j That is, f A (v j ) = a j w + a 2j w a mj w m = m a ij w i i= In other words, the jth column of A describes the image under f A of the vector v j, in terms of the coefficients of f A (v j ) in the basis {w, w 2,, w m } If V and W are spaces of real or complex vectors, then, by convention, the bases {v j } n j= and {w i } m i= are each chosen to be the standard basis for Rn and R m, respectively The jth vector in 4
5 the standard basis is a vector whose components are all zero, except for the jth component, which is equal to one These vectors are called the standard basis vectors of an ndimensional space of real or complex vectors, and are denoted by e j From this point on, we will generally assume that V is R n, and that the field is R, for simplicity Example The standard basis for R 3 consists of the vectors e = 0, 0 e 2 =, e 3 = 0 0 To describe the action of A on a general vector x from V, we can write x = x e + x 2 e x n e n = Then, because A represents a linear function, f A (x) = x j f A (e j ) = j= 0 0 x j e j j= x j a j, where a j is the jth column of A We define the vector y = f A (x) above to be the matrixvector product of A and x, which we denote by Ax Each element of the vector y = Ax is given by j= y i = [Ax i = a i x + a i2 x a in x n = a ij x j From this definition, we see that the jth column of A is equal to the matrixvector product Ae j Example Let Then A = Ax = , x = j= = We see that Ax is a linear combination of the columns of A, with the coefficients of the linear combination obtained from the components of x
6 Matrix Multiplication It follows from this definition that a general system of m linear equations in n unknowns can be described in matrixvector form by the equation Ax = b, where Ax is a matrixvector product of the m n coefficient matrix A and the vector of unknowns x, and b is the vector of righthand side values Of course, if m = n =, the system of equations Ax = b reduces to the scalar linear equation ax = b, which has the solution x = a b, provided that a = 0 As a is the unique number such that a a = aa =, it is desirable to generalize the concepts of multiplication and identity element to square matrices, for which m = n The matrixvector product can be used to define the composition of linear functions represented by matrices Let A be an m n matrix, and let B be an n p matrix Then, if x is a vector of length p, and y = Bx, then we have where C is an m p matrix with entries Ay = A(Bx) = (AB)x = Cx, C ij = a ik b kj k= We define the matrix product of A and B to be the matrix C = AB with entries defined in this manner It should be noted that the product BA is not defined, unless m = p Even if this is the case, in general, AB = BA That is, matrix multiplication is not commutative However, matrix multiplication is associative, meaning that if A is m n, B is n p, and C is p k, then A(BC) = (AB)C Example Consider the 2 2 matrices [ 2 A = 3 4 [ 5 6, B = 7 8 Then AB = [ [ [ ( 5) 2(7) (6) 2( 8) = 3( 5) + 4(7) 3(6) + 4( 8) = [ , whereas BA = [ We see that AB = BA [ = [ 5() + 6( 3) 5( 2) + 6(4) 7() 8( 3) 7( 2) 8(4) = [
7 Vector Operations for Matrices The set of all matrices of size m n, for fixed m and n, is itself a vector space of dimension mn The operations of vector addition and scalar multiplication for matrices are defined as follows: If A and B are m n matrices, then the sum of A and B, denoted by A + B, is the m n matrix C with entries c ij = a ij + b ij If α is a scalar, then the product of α and an m n matrix A, denoted by αa, is the m n matrix B with entries b ij = αa ij It is natural to identify m n matrices with vectors of length mn, in the context of these operations Matrix addition and scalar multiplication have properties analogous to those of vector addition and scalar multiplication In addition, matrix multiplication has the following properties related to these operations We assume that A is an m n matrix, B and D are n k matrices, and α is a scalar Distributivity: A(B + D) = AB + AD Commutativity of scalar multiplication: α(ab) = (αa)b = A(αB) The Transpose of a Matrix An n n matrix A is said to be symmetric if a ij = a ji for i, j =, 2,, n The n n matrix B whose entries are defined by b ij = a ji is called the transpose of A, which we denote by A T Therefore, A is symmetric if A = A T More generally, if A is an m n matrix, then A T is the n n matrix B whose entries are defined by b ij = a ji The transpose has the following properties: (A T ) T = A (A + B) T = A T + B T (AB) T = B T A T Example Let A be the matrix from a previous example, 3 0 A = Then A T =
8 It follows that A + A T = = This matrix is symmetric This can also be seen by the properties of the transpose, since (A + A T ) T = A T + (A T ) T = A T + A = A + A T Other Fundamental Matrix Computations We now define several other operations on matrices and vectors that will be useful in our study of numerical linear algebra For simplicity, we work with real vectors and matrices Given two vectors x and y in R n, the dot product, or inner product, of x and y is the scalar x T y = x y + x 2 y x n y n = x i y i, where x = x x 2 x n, y = Note that x and y must both be defined to be column vectors, and they must have the same length If x T y = 0, then we say that x and y are orthogonal Let x R m and y R n, where m and n are not necessarily equal The term inner product suggests the existence of another operation called the outer product, which is defined by xy T = y y 2 y n i= x y x y 2 x y n x 2 y x 2 y 2 x 2 y n x m y x m y 2 x m y n Note that whereas the inner product is a scalar, the outer product is an m n matrix If x, y R n, the Hadamard product, or componentwise product, of x and y, denoted by x y or x y, is the vector z obtained by multiplying corresponding components of x and y That is, if z = x y, then z i = x i y i, for i =, 2,, n Example If x = [ 2 T [ T and y = 3 4, then [ x T y = ( 3) + ( 2)4 =, xy T ( 3) (4) = 2( 3) 2(4) = [ , 8
9 and x y = [ ( 3) 2(4) = [ 3 8 It is useful to describe matrices as collections of row or column vectors Specifically, a row partition of an m n matrix A is a description of A as a stack of row vectors r T, rt 2,, rt m That is, r T r T A = 2 On the other hand, we can view A is a concatenation of column vectors c, c 2,, c n : r T m A = [ c c 2 c n This description of A is called a column partition Understanding MatrixMatrix Multiplication The fundamental operation of matrixmatrix multiplication can be understood in three different ways, based on other operations that can be performed on matrices and vectors Let A be an m n matrix, and B be an n p matrix, in which case C = AB is an m p matrix We can then view the computation of C in the following ways: Dot product: each entry c ij is the dot product of the ith row of A and the jth column of B Matrixvector multiplication: the jth column of C is a linear combination of the columns of A, where the coefficients are obtained from the jth column of B That is, if C = [ c c 2 c p, B = [ b b 2 b p are column partitions of C and B, then c j = Ab j, for j =, 2,, p Outer product: given the partitions we can write A = [ a a 2 a n, B = C = a b T + a 2 b T a n b T n = That is, C is a sum of outer product updates b T b T 2 b T n, a i b T i i= 9
10 Other Essential Ideas from Linear Algebra We now introduce other important concepts from linear algebra that will prove to be useful in our analysis of algorithms from numerical linear algebra Special Subspaces Let A be an m n matrix Then the range of A, denoted by ran(a), is the set of all vectors of the form y = Ax, where x R n It follows that ran(a) is the span of the columns of A, which is also called the column space of A The dimension of ran(a) is called the column rank of A Similarly, the dimension of the row space of A is called the row rank of A, which is also the column rank of A T It can be shown that the row rank and column rank are equal; this common value is simply called the rank of A, and is denoted by rank(a) We say that A is rankdeficient if rank(a) < min{m, n}; otherwise, we say that A has full rank It is interesting to note that any outer product of vectors has rank one The null space of A, denoted by null(a), is the set of all vectors x R n such that Ax = 0 Its dimension is called the nullity of A It can be shown that for an m n matrix A, The Identity Matrix dim(null(a)) + rank(a) = n When n =, the identity element of matrices, the number, is the unique number such that a() = (a) = a for any number a To determine the identity element for n n matrices, we seek a matrix I such that AI = IA = A for any n n matrix A That is, we must have a ik I kj = a ij, k= i, j =,, n This can only be guaranteed for any matrix A if I jj = for j =, 2,, n, and I ij = 0 when i = j We call this matrix the identity matrix I = Note that the jth column of I is the standard basis vector e j 0
11 The Inverse of a Matrix Given an n n matrix A, it is now natural to ask whether it is possible to find an n n matrix B such that AB = BA = I Such a matrix, if it exists, would then serve as the inverse of A, in the sense of matrix multiplication We denote this matrix by A, just as we denote the multiplicative inverse of a nonzero number a by a If the inverse of A exists, we say that A is invertible or nonsingular; otherwise, we say that A is singular If A exists, then we can use it to describe the solution of the system of linear equations Ax = b, for A Ax = (A A)x = Ix = x = A b, which generalizes the solution x = a b of a single linear equation in one unknown However, just as we can use the inverse to describe the solution to a system of linear equations, we can use systems of linear equations to characterize the inverse Because A satisfies AA = I, it follows from multiplication of both sides of this equation by the jth standard basis vector e j that Ab j = e j, j =, 2,, n, where b j = A e j is the jth column of B = A That is, we can compute A by solving n systems of linear equations of the form Ab j = e j, using a method such as Gaussian elimination and back substitution If Gaussian elimination fails due to the inability to obtain a nonzero pivot element for each column, then A does not exist, and we conclude that A is singular The inverse of a nonsingular matrix A has the following properties: A is unique A is nonsingular, and (A ) = A If B is also a nonsingular n n matrix, then (AB) = B A (A ) T = (A T ) It is common practice to denote the transpose of A by A T Because the set of all n n matrices has an identity element, matrix multiplication is associative, and each nonsingular n n matrix has a unique inverse with respect to matrix multiplication that is also an n n nonsingular matrix, this set forms a group, which is denoted by GL(n), the general linear group The ShermanMorrisonWoodbury Formula Suppose a matrix B has the form B = A + uv T,
12 where A is nonsingular u and v are given vectors This modification of A to obtain B is called a rankone update, since uv T, an outer product, has rank one, due to every vector in the range of uv T being a scalar multiple of u To obtain B from A, we note that if then which yields Ax = u, Bx = (A + uv T )x = ( + v T x)u, B u = + v T A u A u On the other hand, if x is such that v T A x = 0, then which yields BA x = (A + uv T )A x = x, B x = A x This takes us to the following more general problem: given a matrix C, we wish to construct a matrix D such that the following conditions are satisfied: Dw = z, for given vectors w and z Dy = Cy, if y is orthogonal to a given vector g In our application, C = A, D = B, w = u, z = /( + v T A u)a u, and g = A T v To solve this problem, we set (z Cw)gT D = C + g T w Then, if g T y = 0, the second term in the definition of D vanishes, and we obtain Dy = Cy, but in computing Dw, we obtain factors of g T w in the numerator and denominator that cancel, which yields Dw = Cw + (z Cw) = z Applying this definition of D, we obtain ( ) B = A +v + T A u A u A u v T A v T A u = A A uv T A + v T A u This formula for the inverse of a rankone update is known as the ShermanMorrison Formula It is a special case of the ShermanMorrisonWoodbury Formula, (A + UV T ) = A A U(I + V T A U) V T A, where U and V are n k matrices, which means that A + UV T is a rankk update of A 2
13 The Determinant of a Matrix We previously learned that a 2 2 matrix A is invertible if and only if the quantity a a 22 a 2 a 2 is nonzero This generalizes the fact that a matrix a is invertible if and only if its single entry, a = a, is nonzero We now discuss the generalization of this determination of invertibility to general square matrices The determinant of an n n matrix A, denoted by det(a) or A, is defined as follows: If n =, then det(a) = a If n >, then det(a) is recursively defined by det(a) = a ij ( ) i+j det(m ij ), j= i n, where M ij, called a minor of A, is the matrix obtained by removing row i and column j of A Alternatively, det(a) = a ij ( ) i+j det(m ij ), i= j n The matrix A ij = ( ) i+j M ij is called a cofactor of A This definition of the determinant, however, does not lead directly to a practical algorithm for its computation, because it requires O(n!) floatingpoint operations, whereas typical algorithms for matrix computations run in polynomial time However, the computational effort can be reduced by choosing from the multiple formulas for det(a) above By consistently choosing the row or column with the most zeros, the number of operations can be minimized However, more practical methods for computing the determinant can be obtained by using its properties: If any row or column of A has only zero entries, then det(a) = 0 If any two rows or columns of A are the same, then det(a) = 0 If B is an n n matrix, then det(ab) = det(a) det(b) det(a T ) = det(a) If A is nonsingular, then det(a ) = (det(a)) The bestknown application of the determinant is the fact that it indicates whether a matrix A is nonsingular, or invertible The following statements are all equivalent det(a) = 0 3
14 A is nonsingular A exists The system Ax = b has a unique solution for any nvector b The system Ax = 0 has only the trivial solution x = 0 The determinant has other interesting applications The determinant of a 3 3 matrix is equal to the volume of a parallelepiped defined by the vectors that are the rows (or columns) of the matrix This is a special case of the fact that the determinant of an n n matrix is equal to the product of its eigenvalues Differentiation of Matrices Suppose that A(t) is an m n matrix in which each entry is a function of a parameter t Then the matrix A (t), or da/dt, is the m n matrix obtained by differentiating each entry with respect to t That is, [ da(t) = d dt dt [a ij(t) ij Matrices obey differentiation rules that are analogous to differentiation rules for functions, but the rules of matrixmatrix multiplication must be taken into account For example, if A(t) is an m n matrix and B is an n p matrix, then and if A(t) is a nonsingular n n matrix, then d dt [A(t)B(t) = d dt [A(t)B(t) + A(t) d dt [B(t), d dt [A(t) = A(t) d dt [A(t)A(t) It is also useful to know how to differentiate functions of vectors with respect to their components Let A be an n n matrix and x R n Then, we have ( x T Ax ) = (A + A T )x, (x T b) = b These formulas are useful in problems involving minimization of functions of x, such as the leastsquares problem, which entails approximately solving a system of m equations in n unknowns, where typically m > n 4
MAT 610: Numerical Linear Algebra. James V. Lambers
MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic
More informationLinear Algebra and Matrix Inversion
Jim Lambers MAT 46/56 Spring Semester 29 Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much
More informationJim Lambers MAT 419/519 Summer Session Lecture 11 Notes
Jim Lambers MAT 49/59 Summer Session 202 Lecture Notes These notes correspond to Section 34 in the text Broyden s Method One of the drawbacks of using Newton s Method to solve a system of nonlinear equations
More informationMath Linear Algebra Final Exam Review Sheet
Math 151 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a componentwise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationMath 4A Notes. Written by Victoria Kala Last updated June 11, 2017
Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...
More informationCS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
More informationb 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n
Lectures 2: Linear Algebra Background Almost all linear and nonlinear problems in scientific computation require the use of linear algebra These lectures review basic concepts in a way that has proven
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 1: Course Overview; Matrix Multiplication Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More informationChapter 4  MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in
Chapter 4  MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an mbyn array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More informationElementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
More informationLecture Summaries for Linear Algebra M51A
These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture
More informationELE/MCE 503 Linear Algebra Facts Fall 2018
ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2
More informationFoundations of Matrix Analysis
1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationLinear Algebra M1  FIB. Contents: 5. Matrices, systems of linear equations and determinants 6. Vector space 7. Linear maps 8.
Linear Algebra M1  FIB Contents: 5 Matrices, systems of linear equations and determinants 6 Vector space 7 Linear maps 8 Diagonalization Anna de Mier Montserrat Maureso Dept Matemàtica Aplicada II Translation:
More informationMatrices and Linear Algebra
Contents Quantitative methods for Economics and Business University of Ferrara Academic year 20172018 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 1: Course Overview & MatrixVector Multiplication Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 20 Outline 1 Course
More informationMath Bootcamp An pdimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An pdimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationMath 3108: Linear Algebra
Math 3108: Linear Algebra Instructor: Jason Murphy Department of Mathematics and Statistics Missouri University of Science and Technology 1 / 323 Contents. Chapter 1. Slides 3 70 Chapter 2. Slides 71 118
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More information2. Every linear system with the same number of equations as unknowns has a unique solution.
1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations
More informationMaterials engineering Collage \\ Ceramic & construction materials department Numerical Analysis \\Third stage by \\ Dalya Hekmat
Materials engineering Collage \\ Ceramic & construction materials department Numerical Analysis \\Third stage by \\ Dalya Hekmat Linear Algebra Lecture 2 1.3.7 Matrix Matrix multiplication using Falk s
More informationLinear Algebra Highlights
Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationLinear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
More informationProblem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show
MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology  Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,
More informationOnline Exercises for Linear Algebra XM511
This document lists the online exercises for XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Lecture 02 ( 1.1) Online Exercises for Linear Algebra XM511 1) The matrix [3 2
More informationGaussian Elimination and Back Substitution
Jim Lambers MAT 610 Summer Session 200910 Lecture 4 Notes These notes correspond to Sections 31 and 32 in the text Gaussian Elimination and Back Substitution The basic idea behind methods for solving
More informationBasic Concepts in Linear Algebra
Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear
More informationLinear Algebra Formulas. Ben Lee
Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More informationMAT Linear Algebra Collection of sample exams
MAT 342  Linear Algebra Collection of sample exams Ax. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationMatrix operations Linear Algebra with Computer Science Application
Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the
More informationReview of Some Concepts from Linear Algebra: Part 2
Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set
More informationNOTES on LINEAR ALGEBRA 1
School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura
More informationLINEAR ALGEBRA REVIEW
LINEAR ALGEBRA REVIEW SPENCER BECKERKAHN Basic Definitions Domain and Codomain. Let f : X Y be any function. This notation means that X is the domain of f and Y is the codomain of f. This means that for
More informationDigital Workbook for GRA 6035 Mathematics
Eivind Eriksen Digital Workbook for GRA 6035 Mathematics November 10, 2014 BI Norwegian Business School Contents Part I Lectures in GRA6035 Mathematics 1 Linear Systems and Gaussian Elimination........................
More informationLecture Notes in Linear Algebra
Lecture Notes in Linear Algebra Dr. Abdullah AlAzemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................
More informationReview of Basic Concepts in Linear Algebra
Review of Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University September 7, 2017 Math 565 Linear Algebra Review September 7, 2017 1 / 40 Numerical Linear Algebra
More informationChapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015
Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal
More informationLinear Algebra Practice Problems
Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,
More informationIFT 6760A  Lecture 1 Linear Algebra Refresher
IFT 6760A  Lecture 1 Linear Algebra Refresher Scribe(s): Tianyu Li Instructor: Guillaume Rabusseau 1 Summary In the previous lecture we have introduced some applications of linear algebra in machine learning,
More information1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true?
. Let m and n be two natural numbers such that m > n. Which of the following is/are true? (i) A linear system of m equations in n variables is always consistent. (ii) A linear system of n equations in
More informationChapter 1 Vector Spaces
Chapter 1 Vector Spaces PerOlof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 110 Linear Algebra Vector Spaces Definition A vector space V over a field
More informationMatrices and Vectors. Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A =
30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can
More informationChapter 1 Matrices and Systems of Equations
Chapter 1 Matrices and Systems of Equations System of Linear Equations 1. A linear equation in n unknowns is an equation of the form n i=1 a i x i = b where a 1,..., a n, b R and x 1,..., x n are variables.
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661  Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More information1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det
What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix
More information1 Matrices and Systems of Linear Equations. a 1n a 2n
March 31, 2013 161 16. Systems of Linear Equations 1 Matrices and Systems of Linear Equations An m n matrix is an array A = (a ij ) of the form a 11 a 21 a m1 a 1n a 2n... a mn where each a ij is a real
More informationGlossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB
Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the
More informationLinear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations.
POLI 7  Mathematical and Statistical Foundations Prof S Saiegh Fall Lecture Notes  Class 4 October 4, Linear Algebra The analysis of many models in the social sciences reduces to the study of systems
More informationLecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,
2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for
More informationFinal Review Written by Victoria Kala SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015
Final Review Written by Victoria Kala vtkala@mathucsbedu SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015 Summary This review contains notes on sections 44 47, 51 53, 61, 62, 65 For your final,
More informationMAT 2037 LINEAR ALGEBRA I web:
MAT 237 LINEAR ALGEBRA I 2625 Dokuz Eylül University, Faculty of Science, Department of Mathematics web: Instructor: Engin Mermut http://kisideuedutr/enginmermut/ HOMEWORK 2 MATRIX ALGEBRA Textbook: Linear
More informationSection 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.
Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationEcon Slides from Lecture 7
Econ 205 Sobel Econ 205  Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationa11 a A = : a 21 a 22
Matrices The study of linear systems is facilitated by introducing matrices. Matrix theory provides a convenient language and notation to express many of the ideas concisely, and complicated formulas are
More informationChapter 3. Vector spaces
Chapter 3. Vector spaces Lecture notes for MA1111 P. Karageorgis pete@maths.tcd.ie 1/22 Linear combinations Suppose that v 1,v 2,...,v n and v are vectors in R m. Definition 3.1 Linear combination We say
More informationMath 2174: Practice Midterm 1
Math 74: Practice Midterm Show your work and explain your reasoning as appropriate. No calculators. One page of handwritten notes is allowed for the exam, as well as one blank page of scratch paper.. Consider
More informationG1110 & 852G1 Numerical Linear Algebra
The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationChapter 2: Matrices and Linear Systems
Chapter 2: Matrices and Linear Systems Paul Pearson Outline Matrices Linear systems Row operations Inverses Determinants Matrices Definition An m n matrix A = (a ij ) is a rectangular array of real numbers
More informationOHSx XM511 Linear Algebra: Solutions to Online True/False Exercises
This document gives the solutions to all of the online exercises for OHSx XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Answers are in square brackets [. Lecture 02 ( 1.1)
More information1. Linear systems of equations. Chapters 78: Linear Algebra. Solution(s) of a linear system of equations (continued)
1 A linear system of equations of the form Sections 75, 78 & 81 a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2 a m1 x 1 + a m2 x 2 + + a mn x n = b m can be written in matrix
More informationMathematical Foundations
Chapter 1 Mathematical Foundations 1.1 BigO Notations In the description of algorithmic complexity, we often have to use the order notations, often in terms of big O and small o. Loosely speaking, for
More informationReview Questions REVIEW QUESTIONS 71
REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of
More informationMATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.
MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Rowcolumn rule: ijth entry of AB:
More informationDSGA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DSGA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A pdimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationFundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationChapter Two Elements of Linear Algebra
Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationChapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form
Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationMA 265 FINAL EXAM Fall 2012
MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationB553 Lecture 5: Matrix Algebra Review
B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations
More informationUndergraduate Mathematical Economics Lecture 1
Undergraduate Mathematical Economics Lecture 1 Yu Ren WISE, Xiamen University September 15, 2014 Outline 1 Courses Description and Requirement 2 Course Outline ematical techniques used in economics courses
More informationGraduate Mathematical Economics Lecture 1
Graduate Mathematical Economics Lecture 1 Yu Ren WISE, Xiamen University September 23, 2012 Outline 1 2 Course Outline ematical techniques used in graduate level economics courses Mathematics for Economists
More informationMath Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
More information3 (Maths) Linear Algebra
3 (Maths) Linear Algebra References: Simon and Blume, chapters 6 to 11, 16 and 23; Pemberton and Rau, chapters 11 to 13 and 25; Sundaram, sections 1.3 and 1.5. The methods and concepts of linear algebra
More informationReview 1 Math 321: Linear Algebra Spring 2010
Department of Mathematics and Statistics University of New Mexico Review 1 Math 321: Linear Algebra Spring 2010 This is a review for Midterm 1 that will be on Thursday March 11th, 2010. The main topics
More informationMatrices and systems of linear equations
Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra  MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T.
More informationMatrix Algebra for Engineers Jeffrey R. Chasnov
Matrix Algebra for Engineers Jeffrey R. Chasnov The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Department of Mathematics Clear Water Bay, Kowloon
More informationLinear Systems and Matrices
Department of Mathematics The Chinese University of Hong Kong 1 System of m linear equations in n unknowns (linear system) a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.......
More information(b) If a multiple of one row of A is added to another row to produce B then det(b) =det(a).
.(5pts) Let B = 5 5. Compute det(b). (a) (b) (c) 6 (d) (e) 6.(5pts) Determine which statement is not always true for n n matrices A and B. (a) If two rows of A are interchanged to produce B, then det(b)
More informationIntroduction to Matrices
POLS 704 Introduction to Matrices Introduction to Matrices. The Cast of Characters A matrix is a rectangular array (i.e., a table) of numbers. For example, 2 3 X 4 5 6 (4 3) 7 8 9 0 0 0 Thismatrix,with4rowsand3columns,isoforder
More informationMTH 464: Computational Linear Algebra
MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University March 2, 2018 Linear Algebra (MTH 464)
More informationFoundations of Cryptography
Foundations of Cryptography Ville Junnila, Arto Lepistö viljun@utu.fi, alepisto@utu.fi Department of Mathematics and Statistics University of Turku 2017 Ville Junnila, Arto Lepistö viljun@utu.fi, alepisto@utu.fi
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150  Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationMatrices Gaussian elimination Determinants. Graphics 2009/2010, period 1. Lecture 4: matrices
Graphics 2009/2010, period 1 Lecture 4 Matrices m n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in
More informationMath113: Linear Algebra. Beifang Chen
Math3: Linear Algebra Beifang Chen Spring 26 Contents Systems of Linear Equations 3 Systems of Linear Equations 3 Linear Systems 3 2 Geometric Interpretation 3 3 Matrices of Linear Systems 4 4 Elementary
More information