# Jim Lambers MAT 610 Summer Session Lecture 1 Notes

Size: px
Start display at page:

## Transcription

1 Jim Lambers MAT 60 Summer Session Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra by numerical methods that can be implemented on a computer We begin with a brief discussion of the problems that will be discussed in this course, and then establish some fundamental concepts that will serve as the foundation of our discussion of various numerical methods for solving these problems Systems of Linear Equations One of the most fundamental problems in computational mathematics is to solve a system of n linear equations a x + a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2 a m x + a m2 x a mn x n = b m for the n unknowns x, x 2,, x n Many data fitting problems, such as polynomial interpolation or least-squares approximation, involve the solution of such a system Discretization of partial differential equations often yields systems of linear equations that must be solved Systems of nonlinear equations are typically solved using iterative methods that solve a system of linear equations during each iteration In this course, we will study the solution of this type of problem in detail The Eigenvalue Problem As we will see, a system of linear equations can be described in terms of a single vector equation, in which the product of a known matrix of the coefficients a ij, and an unknown vector containing the unknowns x j, is equated to a known vector containing the right-hand side values b i Unlike multiplication of numbers, multiplication of a matrix and a vector is an operation that is difficult to understand intuitively However, given a matrix A, there are certain vectors for which multiplication by A is equivalent to ordinary multiplication This leads to the eigenvalue problem, which consists of finding a nonzero vector x and a number λ such that Ax = λx

2 The number λ is called an eigenvalue of A, and the vector x is called an eigenvector corresponding to λ This is another problem whose solution we will discuss in this course Fundamental Concepts, Algorithms, and Notation Writing a system of equations can be quite tedious Therefore, we instead represent a system of linear equations using a matrix, which is an array of elements, or entries We say that a matrix A is m n if it has m rows and n columns, and we denote the element in row i and column j by a ij We also denote the matrix A by [a ij With this notation, a general system of m equations with n unknowns can be represented using a matrix A that contains the coefficients of the equations, a vectors x that contains the unknowns, and a vector b that contains the quantities on the right-hand sides of the equations Specifically, A = a a 2 a n a 2 a 22 a 2n a m a m2 a mn, x = Note that the vectors x and b are represented by column vectors Vector Spaces and Linear Transformations x x 2 x n, b = Matrices are much more than notational conveniences for writing systems of linear equations A matrix A can also be used to represent a linear function f A whose domain and range are both sets of vectors called vector spaces A vector space over a field (such as the field of real or complex numbers) is a set of vectors, together with two operations: addition of vectors, and multiplication of a vector by a scalar from the field Specifically, if u and v are vectors belonging to a vector space V over a field F, then the sum of u and v, denoted by u + v, is a vector in V, and the scalar product of u with a scalar α in F, denoted by αu, is also a vector in V These operations have the following properties: Commutativity: For any vectors u and v in V, u + v = v + u Associativity: For any vectors u, v and w in V, (u + v) + w = u + (v + w) Identity element for vector addition: There is a vector 0, known as the zero vector, such that for any vector u in V, u + 0 = 0 + u = u b b 2 b m 2

3 Additive inverse: For any vector u in V, there is a unique vector u in V such that u + ( u) = u + u = 0 Distributivity over vector addition: For any vectors u and v in V, and scalar α in F, α(u + v) = αu + αv Distributivity over scalar multiplication: For any vector u in V, and scalars α and β in F, (α + β)u = αu + βu Associativity of scalar multiplication: For any vector u in V and any scalars α and β in F, α(βu) = (αβ)u Identity element for scalar multiplication: For any vector u in V, u = u A function f A : V W, whose domain V and range W are vector spaces over a field F, is a linear transformation if it has the properties f A (x + y) = f A (x) + f A (y), f A (αx) = αf A (x), where x and y are vectors in V and α is a scalar from F Subspaces and Bases Before we can explain how matrices can be used to easily describe linear transformations, we must introduce some important concepts related to vector spaces A subspace of a vector space V is a subset of V that is, itself, a vector space In particular, a subset S of V is also a subsapce if it is closed under the operations of vector addition and scalar multiplication That is, if u and v are vectors in S, then the vectors u + v and αu, where α is any scalar, must also be in S In particular, S cannot be a subspace unless it includes the zero vector Often a vector space or subspace can be characterized as the set of all vectors that can be obtained by adding and/or scaling members of a given set of specific vectors For example, R 2 can be described as the set of all vectors that can be obtained by adding and/or scaling the vectors e = [ 0 [ 0, e 2 = 3

4 These vectors comprise what is known as the standard basis of R 2 More generally, given a set of vectors {v, v 2,, v k } from a vector space V, a vector v V is called a linear combination of v, v 2,, v k if there exist constants c, c 2,, c k such that v = c v + c 2 v c k v k = k c i v i i= We then define the span of {v, v 2,, v k }, denoted by span {v, v 2,, v k }, to be the set of all linear combinations of v, v 2,, v k From the definition of a linear combination, it follows that this set is a subspace of V When a subspace is defined as the span of a set of vectors, it is helpful to know whether the set includes any vectors that are, in some sense, redundant, for if this is the case, the description of the subspace can be simplified To that end, we say that a set of vectors {v, v 2,, v k } is linearly independent if the equation c v + c 2 v c k v k = 0 holds if and only if c = c 2 = = c k = 0 Otherwise, we say that the set is linearly dependent If the set is linearly independent, then any vector v in the span of the set is a unique linear combination of members of the set; that is, there is only one way to choose the coefficients of a linear combination that is used to obtain v Given a vector space V, if there exists a set of vectors {v, v 2,, v k } such that V is the span of {v, v 2,, v k }, and {v, v 2,, v k } is linearly independent, then we say that {v, v 2,, v k } is a basis of V Any basis of V must have the same number of elements, k We call this number the dimension of V, which is denoted by dim(v ) Matrix Representation of Linear Transformations If V is a vector space of dimension n over a field, such as R n or C n, and W is a vector space of dimension m, then a linear function f A with domain V and range W can be represented by an m n matrix A whose entries belong to the field Suppose that the set of vectors {v, v 2,, v n } is a basis for V, and the set {w, w 2,, w m } is a basis for W Then, a ij is the scalar by which w i is multiplied when applying the function f A to the vector v j That is, f A (v j ) = a j w + a 2j w a mj w m = m a ij w i i= In other words, the jth column of A describes the image under f A of the vector v j, in terms of the coefficients of f A (v j ) in the basis {w, w 2,, w m } If V and W are spaces of real or complex vectors, then, by convention, the bases {v j } n j= and {w i } m i= are each chosen to be the standard basis for Rn and R m, respectively The jth vector in 4

5 the standard basis is a vector whose components are all zero, except for the jth component, which is equal to one These vectors are called the standard basis vectors of an n-dimensional space of real or complex vectors, and are denoted by e j From this point on, we will generally assume that V is R n, and that the field is R, for simplicity Example The standard basis for R 3 consists of the vectors e = 0, 0 e 2 =, e 3 = 0 0 To describe the action of A on a general vector x from V, we can write x = x e + x 2 e x n e n = Then, because A represents a linear function, f A (x) = x j f A (e j ) = j= 0 0 x j e j j= x j a j, where a j is the jth column of A We define the vector y = f A (x) above to be the matrix-vector product of A and x, which we denote by Ax Each element of the vector y = Ax is given by j= y i = [Ax i = a i x + a i2 x a in x n = a ij x j From this definition, we see that the jth column of A is equal to the matrix-vector product Ae j Example Let Then A = Ax = , x = j= = We see that Ax is a linear combination of the columns of A, with the coefficients of the linear combination obtained from the components of x

6 Matrix Multiplication It follows from this definition that a general system of m linear equations in n unknowns can be described in matrix-vector form by the equation Ax = b, where Ax is a matrix-vector product of the m n coefficient matrix A and the vector of unknowns x, and b is the vector of right-hand side values Of course, if m = n =, the system of equations Ax = b reduces to the scalar linear equation ax = b, which has the solution x = a b, provided that a = 0 As a is the unique number such that a a = aa =, it is desirable to generalize the concepts of multiplication and identity element to square matrices, for which m = n The matrix-vector product can be used to define the composition of linear functions represented by matrices Let A be an m n matrix, and let B be an n p matrix Then, if x is a vector of length p, and y = Bx, then we have where C is an m p matrix with entries Ay = A(Bx) = (AB)x = Cx, C ij = a ik b kj k= We define the matrix product of A and B to be the matrix C = AB with entries defined in this manner It should be noted that the product BA is not defined, unless m = p Even if this is the case, in general, AB = BA That is, matrix multiplication is not commutative However, matrix multiplication is associative, meaning that if A is m n, B is n p, and C is p k, then A(BC) = (AB)C Example Consider the 2 2 matrices [ 2 A = 3 4 [ 5 6, B = 7 8 Then AB = [ [ [ ( 5) 2(7) (6) 2( 8) = 3( 5) + 4(7) 3(6) + 4( 8) = [ , whereas BA = [ We see that AB = BA [ = [ 5() + 6( 3) 5( 2) + 6(4) 7() 8( 3) 7( 2) 8(4) = [

7 Vector Operations for Matrices The set of all matrices of size m n, for fixed m and n, is itself a vector space of dimension mn The operations of vector addition and scalar multiplication for matrices are defined as follows: If A and B are m n matrices, then the sum of A and B, denoted by A + B, is the m n matrix C with entries c ij = a ij + b ij If α is a scalar, then the product of α and an m n matrix A, denoted by αa, is the m n matrix B with entries b ij = αa ij It is natural to identify m n matrices with vectors of length mn, in the context of these operations Matrix addition and scalar multiplication have properties analogous to those of vector addition and scalar multiplication In addition, matrix multiplication has the following properties related to these operations We assume that A is an m n matrix, B and D are n k matrices, and α is a scalar Distributivity: A(B + D) = AB + AD Commutativity of scalar multiplication: α(ab) = (αa)b = A(αB) The Transpose of a Matrix An n n matrix A is said to be symmetric if a ij = a ji for i, j =, 2,, n The n n matrix B whose entries are defined by b ij = a ji is called the transpose of A, which we denote by A T Therefore, A is symmetric if A = A T More generally, if A is an m n matrix, then A T is the n n matrix B whose entries are defined by b ij = a ji The transpose has the following properties: (A T ) T = A (A + B) T = A T + B T (AB) T = B T A T Example Let A be the matrix from a previous example, 3 0 A = Then A T =

8 It follows that A + A T = = This matrix is symmetric This can also be seen by the properties of the transpose, since (A + A T ) T = A T + (A T ) T = A T + A = A + A T Other Fundamental Matrix Computations We now define several other operations on matrices and vectors that will be useful in our study of numerical linear algebra For simplicity, we work with real vectors and matrices Given two vectors x and y in R n, the dot product, or inner product, of x and y is the scalar x T y = x y + x 2 y x n y n = x i y i, where x = x x 2 x n, y = Note that x and y must both be defined to be column vectors, and they must have the same length If x T y = 0, then we say that x and y are orthogonal Let x R m and y R n, where m and n are not necessarily equal The term inner product suggests the existence of another operation called the outer product, which is defined by xy T = y y 2 y n i= x y x y 2 x y n x 2 y x 2 y 2 x 2 y n x m y x m y 2 x m y n Note that whereas the inner product is a scalar, the outer product is an m n matrix If x, y R n, the Hadamard product, or componentwise product, of x and y, denoted by x y or x y, is the vector z obtained by multiplying corresponding components of x and y That is, if z = x y, then z i = x i y i, for i =, 2,, n Example If x = [ 2 T [ T and y = 3 4, then [ x T y = ( 3) + ( 2)4 =, xy T ( 3) (4) = 2( 3) 2(4) = [ , 8

9 and x y = [ ( 3) 2(4) = [ 3 8 It is useful to describe matrices as collections of row or column vectors Specifically, a row partition of an m n matrix A is a description of A as a stack of row vectors r T, rt 2,, rt m That is, r T r T A = 2 On the other hand, we can view A is a concatenation of column vectors c, c 2,, c n : r T m A = [ c c 2 c n This description of A is called a column partition Understanding Matrix-Matrix Multiplication The fundamental operation of matrix-matrix multiplication can be understood in three different ways, based on other operations that can be performed on matrices and vectors Let A be an m n matrix, and B be an n p matrix, in which case C = AB is an m p matrix We can then view the computation of C in the following ways: Dot product: each entry c ij is the dot product of the ith row of A and the jth column of B Matrix-vector multiplication: the jth column of C is a linear combination of the columns of A, where the coefficients are obtained from the jth column of B That is, if C = [ c c 2 c p, B = [ b b 2 b p are column partitions of C and B, then c j = Ab j, for j =, 2,, p Outer product: given the partitions we can write A = [ a a 2 a n, B = C = a b T + a 2 b T a n b T n = That is, C is a sum of outer product updates b T b T 2 b T n, a i b T i i= 9

10 Other Essential Ideas from Linear Algebra We now introduce other important concepts from linear algebra that will prove to be useful in our analysis of algorithms from numerical linear algebra Special Subspaces Let A be an m n matrix Then the range of A, denoted by ran(a), is the set of all vectors of the form y = Ax, where x R n It follows that ran(a) is the span of the columns of A, which is also called the column space of A The dimension of ran(a) is called the column rank of A Similarly, the dimension of the row space of A is called the row rank of A, which is also the column rank of A T It can be shown that the row rank and column rank are equal; this common value is simply called the rank of A, and is denoted by rank(a) We say that A is rank-deficient if rank(a) < min{m, n}; otherwise, we say that A has full rank It is interesting to note that any outer product of vectors has rank one The null space of A, denoted by null(a), is the set of all vectors x R n such that Ax = 0 Its dimension is called the nullity of A It can be shown that for an m n matrix A, The Identity Matrix dim(null(a)) + rank(a) = n When n =, the identity element of matrices, the number, is the unique number such that a() = (a) = a for any number a To determine the identity element for n n matrices, we seek a matrix I such that AI = IA = A for any n n matrix A That is, we must have a ik I kj = a ij, k= i, j =,, n This can only be guaranteed for any matrix A if I jj = for j =, 2,, n, and I ij = 0 when i = j We call this matrix the identity matrix I = Note that the jth column of I is the standard basis vector e j 0

11 The Inverse of a Matrix Given an n n matrix A, it is now natural to ask whether it is possible to find an n n matrix B such that AB = BA = I Such a matrix, if it exists, would then serve as the inverse of A, in the sense of matrix multiplication We denote this matrix by A, just as we denote the multiplicative inverse of a nonzero number a by a If the inverse of A exists, we say that A is invertible or nonsingular; otherwise, we say that A is singular If A exists, then we can use it to describe the solution of the system of linear equations Ax = b, for A Ax = (A A)x = Ix = x = A b, which generalizes the solution x = a b of a single linear equation in one unknown However, just as we can use the inverse to describe the solution to a system of linear equations, we can use systems of linear equations to characterize the inverse Because A satisfies AA = I, it follows from multiplication of both sides of this equation by the jth standard basis vector e j that Ab j = e j, j =, 2,, n, where b j = A e j is the jth column of B = A That is, we can compute A by solving n systems of linear equations of the form Ab j = e j, using a method such as Gaussian elimination and back substitution If Gaussian elimination fails due to the inability to obtain a nonzero pivot element for each column, then A does not exist, and we conclude that A is singular The inverse of a nonsingular matrix A has the following properties: A is unique A is nonsingular, and (A ) = A If B is also a nonsingular n n matrix, then (AB) = B A (A ) T = (A T ) It is common practice to denote the transpose of A by A T Because the set of all n n matrices has an identity element, matrix multiplication is associative, and each nonsingular n n matrix has a unique inverse with respect to matrix multiplication that is also an n n nonsingular matrix, this set forms a group, which is denoted by GL(n), the general linear group The Sherman-Morrison-Woodbury Formula Suppose a matrix B has the form B = A + uv T,

12 where A is nonsingular u and v are given vectors This modification of A to obtain B is called a rank-one update, since uv T, an outer product, has rank one, due to every vector in the range of uv T being a scalar multiple of u To obtain B from A, we note that if then which yields Ax = u, Bx = (A + uv T )x = ( + v T x)u, B u = + v T A u A u On the other hand, if x is such that v T A x = 0, then which yields BA x = (A + uv T )A x = x, B x = A x This takes us to the following more general problem: given a matrix C, we wish to construct a matrix D such that the following conditions are satisfied: Dw = z, for given vectors w and z Dy = Cy, if y is orthogonal to a given vector g In our application, C = A, D = B, w = u, z = /( + v T A u)a u, and g = A T v To solve this problem, we set (z Cw)gT D = C + g T w Then, if g T y = 0, the second term in the definition of D vanishes, and we obtain Dy = Cy, but in computing Dw, we obtain factors of g T w in the numerator and denominator that cancel, which yields Dw = Cw + (z Cw) = z Applying this definition of D, we obtain ( ) B = A +v + T A u A u A u v T A v T A u = A A uv T A + v T A u This formula for the inverse of a rank-one update is known as the Sherman-Morrison Formula It is a special case of the Sherman-Morrison-Woodbury Formula, (A + UV T ) = A A U(I + V T A U) V T A, where U and V are n k matrices, which means that A + UV T is a rank-k update of A 2

13 The Determinant of a Matrix We previously learned that a 2 2 matrix A is invertible if and only if the quantity a a 22 a 2 a 2 is nonzero This generalizes the fact that a matrix a is invertible if and only if its single entry, a = a, is nonzero We now discuss the generalization of this determination of invertibility to general square matrices The determinant of an n n matrix A, denoted by det(a) or A, is defined as follows: If n =, then det(a) = a If n >, then det(a) is recursively defined by det(a) = a ij ( ) i+j det(m ij ), j= i n, where M ij, called a minor of A, is the matrix obtained by removing row i and column j of A Alternatively, det(a) = a ij ( ) i+j det(m ij ), i= j n The matrix A ij = ( ) i+j M ij is called a cofactor of A This definition of the determinant, however, does not lead directly to a practical algorithm for its computation, because it requires O(n!) floating-point operations, whereas typical algorithms for matrix computations run in polynomial time However, the computational effort can be reduced by choosing from the multiple formulas for det(a) above By consistently choosing the row or column with the most zeros, the number of operations can be minimized However, more practical methods for computing the determinant can be obtained by using its properties: If any row or column of A has only zero entries, then det(a) = 0 If any two rows or columns of A are the same, then det(a) = 0 If B is an n n matrix, then det(ab) = det(a) det(b) det(a T ) = det(a) If A is nonsingular, then det(a ) = (det(a)) The best-known application of the determinant is the fact that it indicates whether a matrix A is nonsingular, or invertible The following statements are all equivalent det(a) = 0 3

14 A is nonsingular A exists The system Ax = b has a unique solution for any n-vector b The system Ax = 0 has only the trivial solution x = 0 The determinant has other interesting applications The determinant of a 3 3 matrix is equal to the volume of a parallelepiped defined by the vectors that are the rows (or columns) of the matrix This is a special case of the fact that the determinant of an n n matrix is equal to the product of its eigenvalues Differentiation of Matrices Suppose that A(t) is an m n matrix in which each entry is a function of a parameter t Then the matrix A (t), or da/dt, is the m n matrix obtained by differentiating each entry with respect to t That is, [ da(t) = d dt dt [a ij(t) ij Matrices obey differentiation rules that are analogous to differentiation rules for functions, but the rules of matrix-matrix multiplication must be taken into account For example, if A(t) is an m n matrix and B is an n p matrix, then and if A(t) is a nonsingular n n matrix, then d dt [A(t)B(t) = d dt [A(t)B(t) + A(t) d dt [B(t), d dt [A(t) = A(t) d dt [A(t)A(t) It is also useful to know how to differentiate functions of vectors with respect to their components Let A be an n n matrix and x R n Then, we have ( x T Ax ) = (A + A T )x, (x T b) = b These formulas are useful in problems involving minimization of functions of x, such as the leastsquares problem, which entails approximately solving a system of m equations in n unknowns, where typically m > n 4

### MAT 610: Numerical Linear Algebra. James V. Lambers

MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic

### Linear Algebra and Matrix Inversion

Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much

### Jim Lambers MAT 419/519 Summer Session Lecture 11 Notes

Jim Lambers MAT 49/59 Summer Session 20-2 Lecture Notes These notes correspond to Section 34 in the text Broyden s Method One of the drawbacks of using Newton s Method to solve a system of nonlinear equations

### Math Linear Algebra Final Exam Review Sheet

Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

### Knowledge Discovery and Data Mining 1 (VO) ( )

Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory

### Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017

Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...

### CS 246 Review of Linear Algebra 01/17/19

1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

### b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n

Lectures -2: Linear Algebra Background Almost all linear and nonlinear problems in scientific computation require the use of linear algebra These lectures review basic concepts in a way that has proven

### AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 1: Course Overview; Matrix Multiplication Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical

### MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

### Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in

Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column

### Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.

Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read

### Elementary maths for GMT

Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

### Lecture Summaries for Linear Algebra M51A

These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture

### ELE/MCE 503 Linear Algebra Facts Fall 2018

ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2

### Foundations of Matrix Analysis

1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the

### EE731 Lecture Notes: Matrix Computations for Signal Processing

EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten

### Linear Algebra M1 - FIB. Contents: 5. Matrices, systems of linear equations and determinants 6. Vector space 7. Linear maps 8.

Linear Algebra M1 - FIB Contents: 5 Matrices, systems of linear equations and determinants 6 Vector space 7 Linear maps 8 Diagonalization Anna de Mier Montserrat Maureso Dept Matemàtica Aplicada II Translation:

### Matrices and Linear Algebra

Contents Quantitative methods for Economics and Business University of Ferrara Academic year 2017-2018 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2

### AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 1: Course Overview & Matrix-Vector Multiplication Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 20 Outline 1 Course

### Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p

Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the

### Math 3108: Linear Algebra

Math 3108: Linear Algebra Instructor: Jason Murphy Department of Mathematics and Statistics Missouri University of Science and Technology 1 / 323 Contents. Chapter 1. Slides 3 70 Chapter 2. Slides 71 118

### Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

### 2. Every linear system with the same number of equations as unknowns has a unique solution.

1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations

### Materials engineering Collage \\ Ceramic & construction materials department Numerical Analysis \\Third stage by \\ Dalya Hekmat

Materials engineering Collage \\ Ceramic & construction materials department Numerical Analysis \\Third stage by \\ Dalya Hekmat Linear Algebra Lecture 2 1.3.7 Matrix Matrix multiplication using Falk s

### Linear Algebra Highlights

Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.

Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the

### Problem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show

MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,

### Online Exercises for Linear Algebra XM511

This document lists the online exercises for XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Lecture 02 ( 1.1) Online Exercises for Linear Algebra XM511 1) The matrix [3 2

### Gaussian Elimination and Back Substitution

Jim Lambers MAT 610 Summer Session 2009-10 Lecture 4 Notes These notes correspond to Sections 31 and 32 in the text Gaussian Elimination and Back Substitution The basic idea behind methods for solving

### Basic Concepts in Linear Algebra

Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear

### Linear Algebra Formulas. Ben Lee

Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries

### 3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions

A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence

### MAT Linear Algebra Collection of sample exams

MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

### Matrix operations Linear Algebra with Computer Science Application

Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the

### Review of Some Concepts from Linear Algebra: Part 2

Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### LINEAR ALGEBRA REVIEW

LINEAR ALGEBRA REVIEW SPENCER BECKER-KAHN Basic Definitions Domain and Codomain. Let f : X Y be any function. This notation means that X is the domain of f and Y is the codomain of f. This means that for

### Digital Workbook for GRA 6035 Mathematics

Eivind Eriksen Digital Workbook for GRA 6035 Mathematics November 10, 2014 BI Norwegian Business School Contents Part I Lectures in GRA6035 Mathematics 1 Linear Systems and Gaussian Elimination........................

### Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

### Review of Basic Concepts in Linear Algebra

Review of Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University September 7, 2017 Math 565 Linear Algebra Review September 7, 2017 1 / 40 Numerical Linear Algebra

### Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

### Linear Algebra Practice Problems

Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,

### IFT 6760A - Lecture 1 Linear Algebra Refresher

IFT 6760A - Lecture 1 Linear Algebra Refresher Scribe(s): Tianyu Li Instructor: Guillaume Rabusseau 1 Summary In the previous lecture we have introduced some applications of linear algebra in machine learning,

### 1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true?

. Let m and n be two natural numbers such that m > n. Which of the following is/are true? (i) A linear system of m equations in n variables is always consistent. (ii) A linear system of n equations in

### Chapter 1 Vector Spaces

Chapter 1 Vector Spaces Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 110 Linear Algebra Vector Spaces Definition A vector space V over a field

### Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =

30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can

### Chapter 1 Matrices and Systems of Equations

Chapter 1 Matrices and Systems of Equations System of Linear Equations 1. A linear equation in n unknowns is an equation of the form n i=1 a i x i = b where a 1,..., a n, b R and x 1,..., x n are variables.

### 2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian

FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian

### 1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det

What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix

### 1 Matrices and Systems of Linear Equations. a 1n a 2n

March 31, 2013 16-1 16. Systems of Linear Equations 1 Matrices and Systems of Linear Equations An m n matrix is an array A = (a ij ) of the form a 11 a 21 a m1 a 1n a 2n... a mn where each a ij is a real

### Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB

Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the

### Linear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations.

POLI 7 - Mathematical and Statistical Foundations Prof S Saiegh Fall Lecture Notes - Class 4 October 4, Linear Algebra The analysis of many models in the social sciences reduces to the study of systems

### Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,

2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for

### Final Review Written by Victoria Kala SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015

Final Review Written by Victoria Kala vtkala@mathucsbedu SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015 Summary This review contains notes on sections 44 47, 51 53, 61, 62, 65 For your final,

### MAT 2037 LINEAR ALGEBRA I web:

MAT 237 LINEAR ALGEBRA I 2625 Dokuz Eylül University, Faculty of Science, Department of Mathematics web: Instructor: Engin Mermut http://kisideuedutr/enginmermut/ HOMEWORK 2 MATRIX ALGEBRA Textbook: Linear

### Section 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.

Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A

### Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

### Econ Slides from Lecture 7

Econ 205 Sobel Econ 205 - Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### a11 a A = : a 21 a 22

Matrices The study of linear systems is facilitated by introducing matrices. Matrix theory provides a convenient language and notation to express many of the ideas concisely, and complicated formulas are

### Chapter 3. Vector spaces

Chapter 3. Vector spaces Lecture notes for MA1111 P. Karageorgis pete@maths.tcd.ie 1/22 Linear combinations Suppose that v 1,v 2,...,v n and v are vectors in R m. Definition 3.1 Linear combination We say

### Math 2174: Practice Midterm 1

Math 74: Practice Midterm Show your work and explain your reasoning as appropriate. No calculators. One page of handwritten notes is allowed for the exam, as well as one blank page of scratch paper.. Consider

### G1110 & 852G1 Numerical Linear Algebra

The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the

### Review of Linear Algebra

Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

### Chapter 2: Matrices and Linear Systems

Chapter 2: Matrices and Linear Systems Paul Pearson Outline Matrices Linear systems Row operations Inverses Determinants Matrices Definition An m n matrix A = (a ij ) is a rectangular array of real numbers

### OHSx XM511 Linear Algebra: Solutions to Online True/False Exercises

This document gives the solutions to all of the online exercises for OHSx XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Answers are in square brackets [. Lecture 02 ( 1.1)

### 1. Linear systems of equations. Chapters 7-8: Linear Algebra. Solution(s) of a linear system of equations (continued)

1 A linear system of equations of the form Sections 75, 78 & 81 a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2 a m1 x 1 + a m2 x 2 + + a mn x n = b m can be written in matrix

### Mathematical Foundations

Chapter 1 Mathematical Foundations 1.1 Big-O Notations In the description of algorithmic complexity, we often have to use the order notations, often in terms of big O and small o. Loosely speaking, for

### Review Questions REVIEW QUESTIONS 71

REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of

### MATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.

MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Row-column rule: i-j-th entry of AB:

### DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

### Introduction to Matrix Algebra

Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

### A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

### Fundamentals of Engineering Analysis (650163)

Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is

### 1. General Vector Spaces

1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

### Chapter Two Elements of Linear Algebra

Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form

Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1

### A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

### MA 265 FINAL EXAM Fall 2012

MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators

### Linear Algebra Primer

Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

### B553 Lecture 5: Matrix Algebra Review

B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations

### Undergraduate Mathematical Economics Lecture 1

Undergraduate Mathematical Economics Lecture 1 Yu Ren WISE, Xiamen University September 15, 2014 Outline 1 Courses Description and Requirement 2 Course Outline ematical techniques used in economics courses

### Graduate Mathematical Economics Lecture 1

Graduate Mathematical Economics Lecture 1 Yu Ren WISE, Xiamen University September 23, 2012 Outline 1 2 Course Outline ematical techniques used in graduate level economics courses Mathematics for Economists

### Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT

Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear

### 3 (Maths) Linear Algebra

3 (Maths) Linear Algebra References: Simon and Blume, chapters 6 to 11, 16 and 23; Pemberton and Rau, chapters 11 to 13 and 25; Sundaram, sections 1.3 and 1.5. The methods and concepts of linear algebra

### Review 1 Math 321: Linear Algebra Spring 2010

Department of Mathematics and Statistics University of New Mexico Review 1 Math 321: Linear Algebra Spring 2010 This is a review for Midterm 1 that will be on Thursday March 11th, 2010. The main topics

### Matrices and systems of linear equations

Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T.

### Matrix Algebra for Engineers Jeffrey R. Chasnov

Matrix Algebra for Engineers Jeffrey R. Chasnov The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Department of Mathematics Clear Water Bay, Kowloon

### Linear Systems and Matrices

Department of Mathematics The Chinese University of Hong Kong 1 System of m linear equations in n unknowns (linear system) a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.......

### (b) If a multiple of one row of A is added to another row to produce B then det(b) =det(a).

.(5pts) Let B = 5 5. Compute det(b). (a) (b) (c) 6 (d) (e) 6.(5pts) Determine which statement is not always true for n n matrices A and B. (a) If two rows of A are interchanged to produce B, then det(b)

### Introduction to Matrices

POLS 704 Introduction to Matrices Introduction to Matrices. The Cast of Characters A matrix is a rectangular array (i.e., a table) of numbers. For example, 2 3 X 4 5 6 (4 3) 7 8 9 0 0 0 Thismatrix,with4rowsand3columns,isoforder

### MTH 464: Computational Linear Algebra

MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University March 2, 2018 Linear Algebra (MTH 464)

### Foundations of Cryptography

Foundations of Cryptography Ville Junnila, Arto Lepistö viljun@utu.fi, alepisto@utu.fi Department of Mathematics and Statistics University of Turku 2017 Ville Junnila, Arto Lepistö viljun@utu.fi, alepisto@utu.fi

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

### Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

### Matrices Gaussian elimination Determinants. Graphics 2009/2010, period 1. Lecture 4: matrices

Graphics 2009/2010, period 1 Lecture 4 Matrices m n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in

### Math113: Linear Algebra. Beifang Chen

Math3: Linear Algebra Beifang Chen Spring 26 Contents Systems of Linear Equations 3 Systems of Linear Equations 3 Linear Systems 3 2 Geometric Interpretation 3 3 Matrices of Linear Systems 4 4 Elementary