Lecture 3: Linear Algebra Review, Part II

Size: px
Start display at page:

Download "Lecture 3: Linear Algebra Review, Part II"

Transcription

1 Lecture 3: Linear Algebra Review, Part II Brian Borchers January 4, Linear Independence Definition The vectors v, v,..., v n are linearly independent if the system of equations c v + c v c n v n = has only the trivial solution c =. If there are multiple solutions, then the vectors are linearly dependent. Determining whether or not a set of vectors is linearly independent is simple just solve the above system of equations. Example Let A = Are the columns of A linearly independent vectors? Setup the system of equations Ax = in an augmented matrix, and then find the RREF. The solutions are x = x 3 We can set x 3 = and obtain the nonzero solution x =. Thus the columns of A are linearly dependent. There are a number of important theoretical consequences of linear independence. For example, it can be shown that if the columns of an n by n matrix A are linearly independent, then A exists, and the system of equations Ax = b has a unique solution for every right hand side b.

2 Subspaces of R n So far, we ve worked with vectors of real numbers in the n dimensional space R n. There are a number of properties of R n that make it convenient to work with vectors. First, the operation of vector addition always works we can take any two vectors in R n and add them together and get another vector in R n. Second, we can multiply any vector in R n by a scalar and obtain another vector in R n. Finally, we have the vector, with the property that for any vector x, x +=+x = x. Definition A subspace W of R n is a subset of R n which satisfies the three properties. If x and y are vectors in W,thenx + y is also a vector in W.. If x is a vector in W and s is any scalar, then sx is also a vector in W. 3. The vector is in W. Example In R 3, the plane P defined by the equation x + x + x 3 = Is a subspace of R n, To see this, note that if we take any two vectors in the plane and add them together, we get another vector in the plane. If we take a vector in this plane and multiply it by any scalar, we get another vector in the plane. Finally, is a vector in the plane. Subspaces are important because they provide an environment within which all of the rules of matrix/vector algebra apply. The most important subspace of R n that we will work with in this course is the null space of a m by n matrix. Definition 3 let A be an m by n matrix. The null space of A (written N(A)) is the set of all vectors x such that Ax =. To show that N(A) is actually a subspace of R n, we need to show three things:. If x and y are in N(A), then Ax =anday =. By adding these equations, we get that A(x + y) =. Thusx + y is in N(A).. If x is in N(A) ands is any scalar, then Ax =. We can multiply this equation by s to get sax =. ThusA(sx) =, and sx is in N(A). 3. A =,soisinn(a). Computationally, finding the null space of a matrix is as simple as solving the system of equations Ax =. Example 3 Let A =

3 In order to find the null space of A, we solve the system of equations Ax =. To solve the equations, we put the system of equations into an augmented matrix and find the RREF 3 From the augmented matrix, we find that x = x x 4 Any vector in the null space can be written as a linear combination of the above vectors, so the null space is a two dimensional plane within R 4. Now, consider the problem of solving Ax = b, where b = 7 39 It happens that one particular solution to this system of equations is p = However, we can take any vector in the null space of A and add it to this solution to obtain another solution. Suppose that x is in N(A). Then A(x + p) =Ax + Ap A(x + p) =+b A(x + p) =b In the context of inverse problems, the null space is critical because the presence of a null space leads to non uniqueness in the solution to a linear system of equations. Definition 4 A basis for a subspace W is a set of vectors v,..., v p such that. Any vector in W can be written as a linear combination of the basis vectors. 3

4 . The basis vectors are linearly independent. Any subspace W of R n will have many different bases. For example, we can take any basis and multiply the basis vectors by to obtain a new basis. However, it is possible to show that all bases for a subspace W have the same number of basis vectors. Definition 5 Let W be a subspace of R n with basis v, v,..., v p. Then the dimension of W is p. Example 4 In the previous example, the vectors v = 3 v = form a basis for the null space of A because any vector in the null space can be written as a linear combination of v and v, and because the vectors v and v are linearly independent. Since the basis has two vectors, the dimension of the null space of A is two. It can be shown that the procedure used in the above example always produces a linearly independent basis for N(A). There is a second important subspace associated with a matrix called the column space of the matrix. Definition 6 let A be an m by n matrix. The column space or range of A (written R(A)) is the set of all vectors b such that Ax = b has at least one solution. In other words, the column space is the set of all vectors b that can be written as a linear combination of the columns of A. The column space is important because it tells us for which vectors b we can solve Ax = b. To find the column space of a matrix, we consider what happens when we compute the RREF of [A b]. In the part of the augmented matrix corresponding to the left hand side of the equations we always get the same result, namely the RREF of A. The solution to the system of equations may involve some free variables, but we can always set these free variables to. Thus when we can solve Ax = b, we can solve the system of equations by using only variables corresponding to the pivot columns in the RREF of A. In other words, if we can solve Ax = b, then we can write b as a linear combination of the pivot columns of A. Note that these are columns from the original matrix A, not columns from the RREF of A. Example 5 As in the previous example, let A = We want to find the column space of A. We already know that the RREF of A 4

5 is 3 Thus whenever we can solve Ax = b, we can find a solution in which x 3 and x 4 are. In other words, whenever there is a solution to Ax = b, we can write b as a linear combination of the first two columns of A b = x x Since these two vectors are linearly independent and span R(A), they form a basis for R(A). The dimension of R(A) is two. In the context of inverse problems, the range of a matrix A is important because it tells us when we can solve Ax = b. In finding the null space and range of a matrix A we found that the basis vectorsforn(a) correspondedto nonpivotcolumns ofa, while the basis vectors for R(A) corresponded to pivot columns of A. Since the matrix A had n columns, we obtain the equation dim N(A) + dim R(A) =n In addition to the null space and range of a matrix A, we will often work with the null space and range of the transpose of A. Since the columns of A T are rows of A, the column space of A T is also called the row space of A. An important theorem of linear algebra is that the dimension of the row space of A equals the dimension of the column space of A. The dimension of the column space of A which is also the dimension of the row space of A is called the rank of A. 3 Orthogonality and the Dot Product Definition 7 Let x and y be two vectors in R n. The dot product of x and y is x y = x T y = x y + x y x n y n Definition 8 Let x be a vector in R n.thelength or norm of x is x = x T x = x + x x n You may be familiar with an alternative definition of the dot product in which x y = x y cos(θ) whereθ is the angle between the two vectors. The two definitions are equivalent. To see this, consider a triangle with sides x, y, and x y. See Figure. The angle between sides x and y is θ. By the law of cosines, x y = x + y x y cos(θ). 5

6 x-y x θ y Figure : Relationship between the dot product and the angle between two vectors. (x y) T (x y) =x T x + y T y x y cos(θ). x T x x T y + y T y = x T x + y T y x y cos(θ). x T y = x y cos(θ). x T y = x y cos(θ). Note that we can also use this formula to compute the angle between two vectors: ( x θ =cos T ) y x y Definition 9 Two vectors x and y in R n are orthogonal or perpendicular if x T y =. Definition A set of vectors v, v,..., v p is orthogonal if each pair of vectors in the set is orthogonal. Definition Two subspaces V and W of R n are orthogonal or perpendicular if every vector in V is perpendicular to every vector in W. It can be shown that N(A) is perpendicular to R(A T ). To see this, note that if x is in N(A), then Ax =. Since the each element of the product Ax is obtained by taking the dot product of a row of A and x, x is perpendicular to each row of A. Since x is perpendicular to all of the columns of A T,itis perpendicular to R(A T ). Definition A basis in which the basis vectors are orthogonal is an orthogonal basis. Definition 3 An n by n matrix Q is orthogonal if the columns of Q are orthogonal and each column of Q has length one. Orthogonal matrices have a number of useful properties:. Q T Q = I. In other words, Q = Q T.. For any vector x in R n, Qx = x. 3. For any two vectors x and y in R n, x T y =(Qx) T (Qy). 6

7 x y p Figure : The orthogonal projection of x onto y. Definition 4 Let A be an m by n matrix. A can be written as A = QR where Q is an m by m orthogonal matrix, and R is an m by n matrix with R ij = whenever i>j. This is called the QR factorization of A. The QR factorization can be easily computed using techniques from numerical linear algebra. If the columns of A are linearly independent, the first n columns of Q are an orthogonal basis for R(A). We can use this approach to find an orthogonal basis for any subspace W of R n. First, we find a basis for W. We put the basis vectors into a matrix, and find its QR factorization. The first columns of the Q matrix form an orthogonal basis for W. Definition 5 Let x and y be two vectors in R n. The orthogonal projection of x onto y is p =proj y x = xt y y T y y See Figure. Definition 6 Let W be a subspace of R n, with an orthogonal basis v, v,..., v p. Let x be a vector in R n. Then the orthogonal projection of x onto W is p =proj W x = xt v v T v v + xt v v T v v xt v p vp T v v p p An important property of the orthogonal projection is that the projection of x onto W is the point in W which is closest to x. In the special case that x is in W, the projection of x onto W is x. This provides a convenient way to write a vector in W as a linear combination of the orthogonal basis vectors. Example 6 In this example, we will find the point on the plane x + x + x 3 = 7

8 which is closest to the point x =,x =,x 3 =3. First, we must find an orthogonal basis for our subspace. Our subspace is the null space of the matrix A = [ ] This null space has the basis u = u = Unfortunately, this basis is not orthogonal. Using the QR factorization, we obtain the orthogonal basis w =.77 w = Using this orthogonal basis, we compute the projection of x onto the plane p = xt w w T w w + xt w w T w w. p =. Given an inconsistent system of equations Ax = b, it is often desirable to find an approximate solution. A natural measure of the quality of an approximate solution is the distance from Ax to b, Ax b. Such a solution is called a least squares solution, because it minimizes the sum of squares of the errors. The least squares solution can be obtained by projecting b onto the range of A. This calculation requires us to first find an orthogonal basis for R(A). There is an alternative approach which does not require the orthogonal basis. Let Ax = p =proj R(A) b. Then Ax b is perpendicular to R(A). In particular, each of the columns of A is orthogonal to Ax b. Thus A T (Ax b) =. A T Ax = A T b. This last system of equations is referred to as the normal equations for the least squares problem. It can be shown that if the columns of A are linearly independent, then the normal equations have exactly one solution. This solution minimizes the sum of squared errors.. 8

9 and Example 7 Let A = b = 3 It s easy to see that the system of equations Ax = b is inconsistent. We ll find the least squares solution by solving the normal equations. [ ] 7 A T A =. 8 A T b = The solution to A T Ax = A T b is [ x = 3 4 [ ]. ]. 9

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP) MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

More information

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures

More information

Section 6.1. Inner Product, Length, and Orthogonality

Section 6.1. Inner Product, Length, and Orthogonality Section 6. Inner Product, Length, and Orthogonality Orientation Almost solve the equation Ax = b Problem: In the real world, data is imperfect. x v u But due to measurement error, the measured x is not

More information

Assignment 1 Math 5341 Linear Algebra Review. Give complete answers to each of the following questions. Show all of your work.

Assignment 1 Math 5341 Linear Algebra Review. Give complete answers to each of the following questions. Show all of your work. Assignment 1 Math 5341 Linear Algebra Review Give complete answers to each of the following questions Show all of your work Note: You might struggle with some of these questions, either because it has

More information

orthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis,

orthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis, 5 Orthogonality Goals: We use scalar products to find the length of a vector, the angle between 2 vectors, projections, orthogonal relations between vectors and subspaces Then we study some applications

More information

March 27 Math 3260 sec. 56 Spring 2018

March 27 Math 3260 sec. 56 Spring 2018 March 27 Math 3260 sec. 56 Spring 2018 Section 4.6: Rank Definition: The row space, denoted Row A, of an m n matrix A is the subspace of R n spanned by the rows of A. We now have three vector spaces associated

More information

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 : Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =

More information

Least squares problems Linear Algebra with Computer Science Application

Least squares problems Linear Algebra with Computer Science Application Linear Algebra with Computer Science Application April 8, 018 1 Least Squares Problems 11 Least Squares Problems What do you do when Ax = b has no solution? Inconsistent systems arise often in applications

More information

Chapter 2 Notes, Linear Algebra 5e Lay

Chapter 2 Notes, Linear Algebra 5e Lay Contents.1 Operations with Matrices..................................1.1 Addition and Subtraction.............................1. Multiplication by a scalar............................ 3.1.3 Multiplication

More information

Math Linear Algebra Final Exam Review Sheet

Math Linear Algebra Final Exam Review Sheet Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

More information

Chapter 6. Orthogonality and Least Squares

Chapter 6. Orthogonality and Least Squares Chapter 6 Orthogonality and Least Squares Section 6.1 Inner Product, Length, and Orthogonality Orientation Recall: This course is about learning to: Solve the matrix equation Ax = b Solve the matrix equation

More information

REVIEW FOR EXAM III SIMILARITY AND DIAGONALIZATION

REVIEW FOR EXAM III SIMILARITY AND DIAGONALIZATION REVIEW FOR EXAM III The exam covers sections 4.4, the portions of 4. on systems of differential equations and on Markov chains, and..4. SIMILARITY AND DIAGONALIZATION. Two matrices A and B are similar

More information

2. Every linear system with the same number of equations as unknowns has a unique solution.

2. Every linear system with the same number of equations as unknowns has a unique solution. 1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations

More information

Chapter 3. Directions: For questions 1-11 mark each statement True or False. Justify each answer.

Chapter 3. Directions: For questions 1-11 mark each statement True or False. Justify each answer. Chapter 3 Directions: For questions 1-11 mark each statement True or False. Justify each answer. 1. (True False) Asking whether the linear system corresponding to an augmented matrix [ a 1 a 2 a 3 b ]

More information

Solutions to Math 51 First Exam April 21, 2011

Solutions to Math 51 First Exam April 21, 2011 Solutions to Math 5 First Exam April,. ( points) (a) Give the precise definition of a (linear) subspace V of R n. (4 points) A linear subspace V of R n is a subset V R n which satisfies V. If x, y V then

More information

MATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.

MATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible. MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Row-column rule: i-j-th entry of AB:

More information

The Four Fundamental Subspaces

The Four Fundamental Subspaces The Four Fundamental Subspaces Introduction Each m n matrix has, associated with it, four subspaces, two in R m and two in R n To understand their relationships is one of the most basic questions in linear

More information

Dot Products, Transposes, and Orthogonal Projections

Dot Products, Transposes, and Orthogonal Projections Dot Products, Transposes, and Orthogonal Projections David Jekel November 13, 2015 Properties of Dot Products Recall that the dot product or standard inner product on R n is given by x y = x 1 y 1 + +

More information

Math 314H EXAM I. 1. (28 points) The row reduced echelon form of the augmented matrix for the system. is the matrix

Math 314H EXAM I. 1. (28 points) The row reduced echelon form of the augmented matrix for the system. is the matrix Math 34H EXAM I Do all of the problems below. Point values for each of the problems are adjacent to the problem number. Calculators may be used to check your answer but not to arrive at your answer. That

More information

YORK UNIVERSITY. Faculty of Science Department of Mathematics and Statistics MATH M Test #1. July 11, 2013 Solutions

YORK UNIVERSITY. Faculty of Science Department of Mathematics and Statistics MATH M Test #1. July 11, 2013 Solutions YORK UNIVERSITY Faculty of Science Department of Mathematics and Statistics MATH 222 3. M Test # July, 23 Solutions. For each statement indicate whether it is always TRUE or sometimes FALSE. Note: For

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

MAT 242 CHAPTER 4: SUBSPACES OF R n

MAT 242 CHAPTER 4: SUBSPACES OF R n MAT 242 CHAPTER 4: SUBSPACES OF R n JOHN QUIGG 1. Subspaces Recall that R n is the set of n 1 matrices, also called vectors, and satisfies the following properties: x + y = y + x x + (y + z) = (x + y)

More information

MATH 323 Linear Algebra Lecture 12: Basis of a vector space (continued). Rank and nullity of a matrix.

MATH 323 Linear Algebra Lecture 12: Basis of a vector space (continued). Rank and nullity of a matrix. MATH 323 Linear Algebra Lecture 12: Basis of a vector space (continued). Rank and nullity of a matrix. Basis Definition. Let V be a vector space. A linearly independent spanning set for V is called a basis.

More information

MATH 1120 (LINEAR ALGEBRA 1), FINAL EXAM FALL 2011 SOLUTIONS TO PRACTICE VERSION

MATH 1120 (LINEAR ALGEBRA 1), FINAL EXAM FALL 2011 SOLUTIONS TO PRACTICE VERSION MATH (LINEAR ALGEBRA ) FINAL EXAM FALL SOLUTIONS TO PRACTICE VERSION Problem (a) For each matrix below (i) find a basis for its column space (ii) find a basis for its row space (iii) determine whether

More information

Elementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th.

Elementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th. Elementary Linear Algebra Review for Exam Exam is Monday, November 6th. The exam will cover sections:.4,..4, 5. 5., 7., the class notes on Markov Models. You must be able to do each of the following. Section.4

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Lecture 13: Orthogonal projections and least squares (Section ) Thang Huynh, UC San Diego 2/9/2018

Lecture 13: Orthogonal projections and least squares (Section ) Thang Huynh, UC San Diego 2/9/2018 Lecture 13: Orthogonal projections and least squares (Section 3.2-3.3) Thang Huynh, UC San Diego 2/9/2018 Orthogonal projection onto subspaces Theorem. Let W be a subspace of R n. Then, each x in R n can

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

MTH 2310, FALL Introduction

MTH 2310, FALL Introduction MTH 2310, FALL 2011 SECTION 6.2: ORTHOGONAL SETS Homework Problems: 1, 5, 9, 13, 17, 21, 23 1, 27, 29, 35 1. Introduction We have discussed previously the benefits of having a set of vectors that is linearly

More information

This lecture is a review for the exam. The majority of the exam is on what we ve learned about rectangular matrices.

This lecture is a review for the exam. The majority of the exam is on what we ve learned about rectangular matrices. Exam review This lecture is a review for the exam. The majority of the exam is on what we ve learned about rectangular matrices. Sample question Suppose u, v and w are non-zero vectors in R 7. They span

More information

ECS130 Scientific Computing. Lecture 1: Introduction. Monday, January 7, 10:00 10:50 am

ECS130 Scientific Computing. Lecture 1: Introduction. Monday, January 7, 10:00 10:50 am ECS130 Scientific Computing Lecture 1: Introduction Monday, January 7, 10:00 10:50 am About Course: ECS130 Scientific Computing Professor: Zhaojun Bai Webpage: http://web.cs.ucdavis.edu/~bai/ecs130/ Today

More information

Math 3191 Applied Linear Algebra

Math 3191 Applied Linear Algebra Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have

More information

A SHORT SUMMARY OF VECTOR SPACES AND MATRICES

A SHORT SUMMARY OF VECTOR SPACES AND MATRICES A SHORT SUMMARY OF VECTOR SPACES AND MATRICES This is a little summary of some of the essential points of linear algebra we have covered so far. If you have followed the course so far you should have no

More information

LINEAR ALGEBRA SUMMARY SHEET.

LINEAR ALGEBRA SUMMARY SHEET. LINEAR ALGEBRA SUMMARY SHEET RADON ROSBOROUGH https://intuitiveexplanationscom/linear-algebra-summary-sheet/ This document is a concise collection of many of the important theorems of linear algebra, organized

More information

Chapter 6 - Orthogonality

Chapter 6 - Orthogonality Chapter 6 - Orthogonality Maggie Myers Robert A. van de Geijn The University of Texas at Austin Orthogonality Fall 2009 http://z.cs.utexas.edu/wiki/pla.wiki/ 1 Orthogonal Vectors and Subspaces http://z.cs.utexas.edu/wiki/pla.wiki/

More information

1 Last time: inverses

1 Last time: inverses MATH Linear algebra (Fall 8) Lecture 8 Last time: inverses The following all mean the same thing for a function f : X Y : f is invertible f is one-to-one and onto 3 For each b Y there is exactly one a

More information

Solutions to Math 51 Midterm 1 July 6, 2016

Solutions to Math 51 Midterm 1 July 6, 2016 Solutions to Math 5 Midterm July 6, 26. (a) (6 points) Find an equation (of the form ax + by + cz = d) for the plane P in R 3 passing through the points (, 2, ), (2,, ), and (,, ). We first compute two

More information

4.3 - Linear Combinations and Independence of Vectors

4.3 - Linear Combinations and Independence of Vectors - Linear Combinations and Independence of Vectors De nitions, Theorems, and Examples De nition 1 A vector v in a vector space V is called a linear combination of the vectors u 1, u,,u k in V if v can be

More information

Overview. Motivation for the inner product. Question. Definition

Overview. Motivation for the inner product. Question. Definition Overview Last time we studied the evolution of a discrete linear dynamical system, and today we begin the final topic of the course (loosely speaking) Today we ll recall the definition and properties of

More information

The set of all solutions to the homogeneous equation Ax = 0 is a subspace of R n if A is m n.

The set of all solutions to the homogeneous equation Ax = 0 is a subspace of R n if A is m n. 0 Subspaces (Now, we are ready to start the course....) Definitions: A linear combination of the vectors v, v,..., v m is any vector of the form c v + c v +... + c m v m, where c,..., c m R. A subset V

More information

LECTURES 14/15: LINEAR INDEPENDENCE AND BASES

LECTURES 14/15: LINEAR INDEPENDENCE AND BASES LECTURES 14/15: LINEAR INDEPENDENCE AND BASES MA1111: LINEAR ALGEBRA I, MICHAELMAS 2016 1. Linear Independence We have seen in examples of span sets of vectors that sometimes adding additional vectors

More information

Math 54 HW 4 solutions

Math 54 HW 4 solutions Math 54 HW 4 solutions 2.2. Section 2.2 (a) False: Recall that performing a series of elementary row operations A is equivalent to multiplying A by a series of elementary matrices. Suppose that E,...,

More information

( v 1 + v 2 ) + (3 v 1 ) = 4 v 1 + v 2. and ( 2 v 2 ) + ( v 1 + v 3 ) = v 1 2 v 2 + v 3, for instance.

( v 1 + v 2 ) + (3 v 1 ) = 4 v 1 + v 2. and ( 2 v 2 ) + ( v 1 + v 3 ) = v 1 2 v 2 + v 3, for instance. 4.2. Linear Combinations and Linear Independence If we know that the vectors v 1, v 2,..., v k are are in a subspace W, then the Subspace Test gives us more vectors which must also be in W ; for instance,

More information

Lecture 9: Vector Algebra

Lecture 9: Vector Algebra Lecture 9: Vector Algebra Linear combination of vectors Geometric interpretation Interpreting as Matrix-Vector Multiplication Span of a set of vectors Vector Spaces and Subspaces Linearly Independent/Dependent

More information

Answer Key for Exam #2

Answer Key for Exam #2 . Use elimination on an augmented matrix: Answer Key for Exam # 4 4 8 4 4 4 The fourth column has no pivot, so x 4 is a free variable. The corresponding system is x + x 4 =, x =, x x 4 = which we solve

More information

February 20 Math 3260 sec. 56 Spring 2018

February 20 Math 3260 sec. 56 Spring 2018 February 20 Math 3260 sec. 56 Spring 2018 Section 2.2: Inverse of a Matrix Consider the scalar equation ax = b. Provided a 0, we can solve this explicity x = a 1 b where a 1 is the unique number such that

More information

Worksheet for Lecture 25 Section 6.4 Gram-Schmidt Process

Worksheet for Lecture 25 Section 6.4 Gram-Schmidt Process Worksheet for Lecture Name: Section.4 Gram-Schmidt Process Goal For a subspace W = Span{v,..., v n }, we want to find an orthonormal basis of W. Example Let W = Span{x, x } with x = and x =. Give an orthogonal

More information

MAT Linear Algebra Collection of sample exams

MAT Linear Algebra Collection of sample exams MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

More information

Solutions to Final Exam

Solutions to Final Exam Solutions to Final Exam. Let A be a 3 5 matrix. Let b be a nonzero 5-vector. Assume that the nullity of A is. (a) What is the rank of A? 3 (b) Are the rows of A linearly independent? (c) Are the columns

More information

MAT2342 : Introduction to Applied Linear Algebra Mike Newman, fall Projections. introduction

MAT2342 : Introduction to Applied Linear Algebra Mike Newman, fall Projections. introduction MAT4 : Introduction to Applied Linear Algebra Mike Newman fall 7 9. Projections introduction One reason to consider projections is to understand approximate solutions to linear systems. A common example

More information

EE263: Introduction to Linear Dynamical Systems Review Session 2

EE263: Introduction to Linear Dynamical Systems Review Session 2 EE263: Introduction to Linear Dynamical Systems Review Session 2 Basic concepts from linear algebra nullspace range rank and conservation of dimension EE263 RS2 1 Prerequisites We assume that you are familiar

More information

Span and Linear Independence

Span and Linear Independence Span and Linear Independence It is common to confuse span and linear independence, because although they are different concepts, they are related. To see their relationship, let s revisit the previous

More information

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear

More information

Orthogonality. Orthonormal Bases, Orthogonal Matrices. Orthogonality

Orthogonality. Orthonormal Bases, Orthogonal Matrices. Orthogonality Orthonormal Bases, Orthogonal Matrices The Major Ideas from Last Lecture Vector Span Subspace Basis Vectors Coordinates in different bases Matrix Factorization (Basics) The Major Ideas from Last Lecture

More information

Miderm II Solutions To find the inverse we row-reduce the augumented matrix [I A]. In our case, we row reduce

Miderm II Solutions To find the inverse we row-reduce the augumented matrix [I A]. In our case, we row reduce Miderm II Solutions Problem. [8 points] (i) [4] Find the inverse of the matrix A = To find the inverse we row-reduce the augumented matrix [I A]. In our case, we row reduce We have A = 2 2 (ii) [2] Possibly

More information

Math 302 Test 1 Review

Math 302 Test 1 Review Math Test Review. Given two points in R, x, y, z and x, y, z, show the point x + x, y + y, z + z is on the line between these two points and is the same distance from each of them. The line is rt x, y,

More information

Lecture 03. Math 22 Summer 2017 Section 2 June 26, 2017

Lecture 03. Math 22 Summer 2017 Section 2 June 26, 2017 Lecture 03 Math 22 Summer 2017 Section 2 June 26, 2017 Just for today (10 minutes) Review row reduction algorithm (40 minutes) 1.3 (15 minutes) Classwork Review row reduction algorithm Review row reduction

More information

MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix

MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix Definition: Let L : V 1 V 2 be a linear operator. The null space N (L) of L is the subspace of V 1 defined by N (L) = {x

More information

Review Let A, B, and C be matrices of the same size, and let r and s be scalars. Then

Review Let A, B, and C be matrices of the same size, and let r and s be scalars. Then 1 Sec 21 Matrix Operations Review Let A, B, and C be matrices of the same size, and let r and s be scalars Then (i) A + B = B + A (iv) r(a + B) = ra + rb (ii) (A + B) + C = A + (B + C) (v) (r + s)a = ra

More information

Exercises Chapter II.

Exercises Chapter II. Page 64 Exercises Chapter II. 5. Let A = (1, 2) and B = ( 2, 6). Sketch vectors of the form X = c 1 A + c 2 B for various values of c 1 and c 2. Which vectors in R 2 can be written in this manner? B y

More information

Mathematical foundations - linear algebra

Mathematical foundations - linear algebra Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar

More information

LINEAR ALGEBRA REVIEW

LINEAR ALGEBRA REVIEW LINEAR ALGEBRA REVIEW When we define a term, we put it in boldface. This is a very compressed review; please read it very carefully and be sure to ask questions on parts you aren t sure of. x 1 WedenotethesetofrealnumbersbyR.

More information

Worksheet for Lecture 23 (due December 4) Section 6.1 Inner product, length, and orthogonality

Worksheet for Lecture 23 (due December 4) Section 6.1 Inner product, length, and orthogonality Worksheet for Lecture (due December 4) Name: Section 6 Inner product, length, and orthogonality u Definition Let u = u n product or dot product to be and v = v v n be vectors in R n We define their inner

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

Math 416, Spring 2010 More on Algebraic and Geometric Properties January 21, 2010 MORE ON ALGEBRAIC AND GEOMETRIC PROPERTIES

Math 416, Spring 2010 More on Algebraic and Geometric Properties January 21, 2010 MORE ON ALGEBRAIC AND GEOMETRIC PROPERTIES Math 46, Spring 2 More on Algebraic and Geometric Properties January 2, 2 MORE ON ALGEBRAIC AND GEOMETRIC PROPERTIES Algebraic properties Algebraic properties of matrix/vector multiplication Last time

More information

MATH 221: SOLUTIONS TO SELECTED HOMEWORK PROBLEMS

MATH 221: SOLUTIONS TO SELECTED HOMEWORK PROBLEMS MATH 221: SOLUTIONS TO SELECTED HOMEWORK PROBLEMS 1. HW 1: Due September 4 1.1.21. Suppose v, w R n and c is a scalar. Prove that Span(v + cw, w) = Span(v, w). We must prove two things: that every element

More information

Announcements Wednesday, November 15

Announcements Wednesday, November 15 3π 4 Announcements Wednesday, November 15 Reviews today: Recitation Style Solve and discuss Practice problems in groups Preparing for the exam tips and strategies It is not mandatory Eduardo at Culc 141,

More information

The definition of a vector space (V, +, )

The definition of a vector space (V, +, ) The definition of a vector space (V, +, ) 1. For any u and v in V, u + v is also in V. 2. For any u and v in V, u + v = v + u. 3. For any u, v, w in V, u + ( v + w) = ( u + v) + w. 4. There is an element

More information

Study Guide for Linear Algebra Exam 2

Study Guide for Linear Algebra Exam 2 Study Guide for Linear Algebra Exam 2 Term Vector Space Definition A Vector Space is a nonempty set V of objects, on which are defined two operations, called addition and multiplication by scalars (real

More information

Review Notes for Linear Algebra True or False Last Updated: February 22, 2010

Review Notes for Linear Algebra True or False Last Updated: February 22, 2010 Review Notes for Linear Algebra True or False Last Updated: February 22, 2010 Chapter 4 [ Vector Spaces 4.1 If {v 1,v 2,,v n } and {w 1,w 2,,w n } are linearly independent, then {v 1 +w 1,v 2 +w 2,,v n

More information

Orthogonality and Least Squares

Orthogonality and Least Squares 6 Orthogonality and Least Squares 6.1 INNER PRODUCT, LENGTH, AND ORTHOGONALITY INNER PRODUCT If u and v are vectors in, then we regard u and v as matrices. n 1 n The transpose u T is a 1 n matrix, and

More information

Final Review Sheet. B = (1, 1 + 3x, 1 + x 2 ) then 2 + 3x + 6x 2

Final Review Sheet. B = (1, 1 + 3x, 1 + x 2 ) then 2 + 3x + 6x 2 Final Review Sheet The final will cover Sections Chapters 1,2,3 and 4, as well as sections 5.1-5.4, 6.1-6.2 and 7.1-7.3 from chapters 5,6 and 7. This is essentially all material covered this term. Watch

More information

Math 51, Homework-2 Solutions

Math 51, Homework-2 Solutions SSEA Summer 27 Math 5, Homework-2 Solutions Write the parametric equation of the plane that contains the following point and line: 3 2, 4 2 + t 3 t R 5 4 By substituting t = and t =, we get two points

More information

1 Last time: multiplying vectors matrices

1 Last time: multiplying vectors matrices MATH Linear algebra (Fall 7) Lecture Last time: multiplying vectors matrices Given a matrix A = a a a n a a a n and a vector v = a m a m a mn Av = v a a + v a a v v + + Rn we define a n a n a m a m a mn

More information

Linear independence, span, basis, dimension - and their connection with linear systems

Linear independence, span, basis, dimension - and their connection with linear systems Linear independence span basis dimension - and their connection with linear systems Linear independence of a set of vectors: We say the set of vectors v v..v k is linearly independent provided c v c v..c

More information

Solutions to Math 51 First Exam October 13, 2015

Solutions to Math 51 First Exam October 13, 2015 Solutions to Math First Exam October 3, 2. (8 points) (a) Find an equation for the plane in R 3 that contains both the x-axis and the point (,, 2). The equation should be of the form ax + by + cz = d.

More information

Announcements Monday, October 29

Announcements Monday, October 29 Announcements Monday, October 29 WeBWorK on determinents due on Wednesday at :59pm. The quiz on Friday covers 5., 5.2, 5.3. My office is Skiles 244 and Rabinoffice hours are: Mondays, 2 pm; Wednesdays,

More information

Multiple Choice Questions

Multiple Choice Questions Multiple Choice Questions There is no penalty for guessing. Three points per question, so a total of 48 points for this section.. What is the complete relationship between homogeneous linear systems of

More information

Topic 14 Notes Jeremy Orloff

Topic 14 Notes Jeremy Orloff Topic 4 Notes Jeremy Orloff 4 Row reduction and subspaces 4. Goals. Be able to put a matrix into row reduced echelon form (RREF) using elementary row operations.. Know the definitions of null and column

More information

Vector Spaces, Orthogonality, and Linear Least Squares

Vector Spaces, Orthogonality, and Linear Least Squares Week Vector Spaces, Orthogonality, and Linear Least Squares. Opening Remarks.. Visualizing Planes, Lines, and Solutions Consider the following system of linear equations from the opener for Week 9: χ χ

More information

Matrix-Vector Products and the Matrix Equation Ax = b

Matrix-Vector Products and the Matrix Equation Ax = b Matrix-Vector Products and the Matrix Equation Ax = b A. Havens Department of Mathematics University of Massachusetts, Amherst January 31, 2018 Outline 1 Matrices Acting on Vectors Linear Combinations

More information

22m:033 Notes: 6.1 Inner Product, Length and Orthogonality

22m:033 Notes: 6.1 Inner Product, Length and Orthogonality m:033 Notes: 6. Inner Product, Length and Orthogonality Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman April, 00 The inner product Arithmetic is based on addition and

More information

Orthogonality. 6.1 Orthogonal Vectors and Subspaces. Chapter 6

Orthogonality. 6.1 Orthogonal Vectors and Subspaces. Chapter 6 Chapter 6 Orthogonality 6.1 Orthogonal Vectors and Subspaces Recall that if nonzero vectors x, y R n are linearly independent then the subspace of all vectors αx + βy, α, β R (the space spanned by x and

More information

PRACTICE PROBLEMS FOR THE FINAL

PRACTICE PROBLEMS FOR THE FINAL PRACTICE PROBLEMS FOR THE FINAL Here are a slew of practice problems for the final culled from old exams:. Let P be the vector space of polynomials of degree at most. Let B = {, (t ), t + t }. (a) Show

More information

Linear Algebra Highlights

Linear Algebra Highlights Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to

More information

Abstract Vector Spaces and Concrete Examples

Abstract Vector Spaces and Concrete Examples LECTURE 18 Abstract Vector Spaces and Concrete Examples Our discussion of linear algebra so far has been devoted to discussing the relations between systems of linear equations, matrices, and vectors.

More information

MATH 1553, SPRING 2018 SAMPLE MIDTERM 2 (VERSION B), 1.7 THROUGH 2.9

MATH 1553, SPRING 2018 SAMPLE MIDTERM 2 (VERSION B), 1.7 THROUGH 2.9 MATH 155, SPRING 218 SAMPLE MIDTERM 2 (VERSION B), 1.7 THROUGH 2.9 Name Section 1 2 4 5 Total Please read all instructions carefully before beginning. Each problem is worth 1 points. The maximum score

More information

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the

More information

Linear Algebra: Sample Questions for Exam 2

Linear Algebra: Sample Questions for Exam 2 Linear Algebra: Sample Questions for Exam 2 Instructions: This is not a comprehensive review: there are concepts you need to know that are not included. Be sure you study all the sections of the book and

More information

Chapter 6: Orthogonality

Chapter 6: Orthogonality Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products

More information

CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33

CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33 Linear Algebra 1/33 Vectors A vector is a magnitude and a direction Magnitude = v Direction Also known as norm, length Represented by unit vectors (vectors with a length of 1 that point along distinct

More information

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017 Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...

More information

Dot product and linear least squares problems

Dot product and linear least squares problems Dot product and linear least squares problems Dot Product For vectors u,v R n we define the dot product Note that we can also write this as u v = u,,u n u v = u v + + u n v n v v n = u v + + u n v n The

More information

Elementary maths for GMT

Elementary maths for GMT Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

More information

Solutions to Final Practice Problems Written by Victoria Kala Last updated 12/5/2015

Solutions to Final Practice Problems Written by Victoria Kala Last updated 12/5/2015 Solutions to Final Practice Problems Written by Victoria Kala vtkala@math.ucsb.edu Last updated /5/05 Answers This page contains answers only. See the following pages for detailed solutions. (. (a x. See

More information

Chapter 2: Matrix Algebra

Chapter 2: Matrix Algebra Chapter 2: Matrix Algebra (Last Updated: October 12, 2016) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). Write A = 1. Matrix operations [a 1 a n. Then entry

More information

7. Dimension and Structure.

7. Dimension and Structure. 7. Dimension and Structure 7.1. Basis and Dimension Bases for Subspaces Example 2 The standard unit vectors e 1, e 2,, e n are linearly independent, for if we write (2) in component form, then we obtain

More information

Section 2.2: The Inverse of a Matrix

Section 2.2: The Inverse of a Matrix Section 22: The Inverse of a Matrix Recall that a linear equation ax b, where a and b are scalars and a 0, has the unique solution x a 1 b, where a 1 is the reciprocal of a From this result, it is natural

More information

2018 Fall 2210Q Section 013 Midterm Exam II Solution

2018 Fall 2210Q Section 013 Midterm Exam II Solution 08 Fall 0Q Section 0 Midterm Exam II Solution True or False questions points 0 0 points) ) Let A be an n n matrix. If the equation Ax b has at least one solution for each b R n, then the solution is unique

More information

MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian.

MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian. MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian. Spanning set Let S be a subset of a vector space V. Definition. The span of the set S is the smallest subspace W V that contains S. If

More information