Lecture 9: Vector Algebra Linear combination of vectors Geometric interpretation Interpreting as Matrix-Vector Multiplication Span of a set of vectors Vector Spaces and Subspaces Linearly Independent/Dependent set of vectors and bases Spanning set Dimension 1
Linear Combination of Vectors Let v = v 1 v 2 v 3. v p R p and c 1, c 2,, c p be scalars then vector y defined by: is called a linear combination of v 1, v 2,. v p with weights c 1, c 2, c p. The weights in a linear combination can be any real numbers including zero. And y is said to be linearly dependent on v 1, v 2,. v p.
Linear Dependence Definition: v 1, v 2, vn are linearly dependent if vi can be written down as a linear combination of rest of the prceeding vectors for any i. v i = c 1 v 1 + c 2 v 2 + c 3 v 3 +.. c i 1 v i 1 for any i = 1,2.. r A zero vector is linearly dependent on any set of N-dimensional vectors where x 1, x 2, are all 0.
Linear Dependence If there is a solution for the vector equation In other words, there exists values for a 1, a 2, a n such that the above equation is satisfied, then b is linearly dependent on x 1, x 2, x n.
Linear Independence If there is no value for a 1, a 2, exists such that the below equation is satisfied then b is linearly independent of the set of vectors v 1, v 2, v n. In other words, x 1 a 1 + x 2 a 2 + x n a n b for any value of x 1, x 2, or x 1 a 1 + x 2 a 2 + x n a n b 0 means b is linearly independent of the set of vectors a 1, a 2, a n In general, the set of vectors {a 1, a 2, a n, b} is linearly independent if x 1 a 1 + x 2 a 2 + x n a n + x n+1 b = 0, only if x 1, x 2, are all 0.
Linear Independence
Linearly Independent/Dependent Set A set of vectors that are linearly independent is called a Linearly Independent Set. If at least one vector in a set of vectors is linearly dependent on other vectors, then that set is called a Linearly Dependent Set. A set of vectors with a zero vector has to be a Linearly Dependent Set..
Examples of Linear Dependence A vector and its scalar multiple are linear dependent. (By definition.) x 1 a = b A vector that is a linear combination of a set of vectors is linearly dependent on those vectors. (By definition.) 8
Examples of Linear Dependence Let x1 = 1 2 3, x2 = 2 4 5 and b = [4 8 11] Then the rank of the matrix M is M = 1 2 3 2 4 5 4 8 11 - the number of linearly independent vectors
Span of a set of vectors A vector that is in the span of a set of vectors is linearly dependent on those vectors. (By definition.)
Span of a set of vectors Example 1:
Span of a set of vectors Example 2: 0 P 2 v 2 v 1 P 1
Same R 2 plane can be spanned by other vectors too Let a= 2 1 and b= 1 2. Any point in R 2 can be represented as a linear combination of the a and b axes. P arbitrary = c 1 a + c 2 b Example: a-coordinate P 1 = 2 0 = 4 2 a + b 3 3 P 2 = 1 2 = 4 a + 5 b 3 3 a and b vectors SPAN the R 2 plane. b-coordinate 5 3 b P 2 4 3 a b a P 1 4 3 a 2 3 b 13
A GEOMETRIC DESCRIPTION OF SPAN V Span of a set of vectors (geometric description) Let v be a nonzero vector in R 3. Then Span v is the set of all scalar multiples of v, which is the set of points on the line in R 3 through v and 0. See the figure below
Span A GEOMETRIC of a set of DESCRIPTION vectors (geometric OF SPAN U, Vdescription) If u and v are nonzero vectors in R 3, with v not a multiple of u, then Span u, v is the plane in R 3 that contains u, v, and 0. In particular, Span u, v contains the line in R 3 through u and 0 and the line through v and 0. See the figure below.
How many minimum number of vectors are necessary to span a space? ONE non-zero vector is required and sufficient to span a 1D space (in other words, a line). The dimension of the vectors has to be 1 or more. TWO linearly independent vectors are required and sufficient to span a 2D plane such as the XY plane. The dimension of the vectors has to be 2 or more. THREE linearly independent vectors are required and sufficient to span a 3D space (such as XYZ volume). The dimension of the vectors has to be 3 or more. N linearly independent vectors are required and sufficient to span an N-D space. The dimension of the vectors has to be N or more. The span of these vectors is called the vector space (or subspace, if it is a subset of a vector space). (Later we will see that a vector space is more generic than a span. But in this course, we will only see vector spaces that are spans of a sets of vectors.) 16
Span of a set of vectors Maximal subset consisting of linearly independent vectors is called the basis of the span of x 1, x 2, x 3, x n And the number of elements in the basis is called the dimension of the span.
Spanning Set A basis is an efficient spanning set that contains no unnecessary vectors. A basis can be constructed from a spanning set by discarding unneeded vectors Basis construction
Spanning Set v 4 =2v 1 v 2. So, by Spanning Set Theorem, {v 1, v 2, v 3 } span the same subspace as {v 1, v 2, v 3, v 4 }. We also see that {v 1, v 2, v 3 } are linearly independent. So that is the basis of W.
Vector Spaces Span of vectors also form a vector space. Now we will look more into the definition of vector spaces and their properties.
Vector Spaces 7) c u + v = cu + cv for any scalar c
Vector Spaces
Vector Subspace If a subset of a vector space also forms a vector space, then that subset is called a vector subspace. Hence, every vector subspace is also a vector space. And also every vector space is also a vector subspace (of itself and possibly of larger spaces).
Vector Subspace
Vector Subspace Example
Vector Subspace Example
Vector Basis
Vector Basis Example: No, because the Span{v 1, v 2 } H. Span{v 1, v 2 } is the entire XY plane, not just the vectors of the form [s,s,0].
Spanning Set The following three sets in R3 show how linearly independent set can be enlarged to a basis and how further enlargement destroys the linear independence of the set.
Dimension of a Vector Space We already discussed Dimension of a vector: Number of components of the vector Dimension of a vector space (or subspace): Minimum number of linearly independent vectors required to span that space. New definition Dimension of a vector space (or subspace): Number of vectors in its basis. What is Basis of vector space again? A Linearly Independent Set (of vectors) whose span is the vector space. This implies: Basis has the minimum number of linearly independent vectors that spans the space. Note: Basis is not unique. But all bases that span the same space has the same number of vectors. 30
Matrix equation Solving a system of m equation and n unknown in Algebra can be converted in matrixvector multiplication form. a 11 x 1 + a 12 x 2 + a 13 x 3 = b 1 a 21 x 1 + a 22 x 2 + a 23 x 3 = b 2 a 31 x 1 + a 32 x 2 + a 33 x 3 = b 3 A is a 3 3 matrix, x and b are 3 1 column vectors. A and b are known, where x is unknown. This is a matrix equation. Ax = b, a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 x 1 x 2 x 3 = b 1 b 2 b 3
Matrix equation as a vector equation Matrix equation can be interpreted as a vector equation. The left side of equation (Ax) can be written as: a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 x 1 x 2 x 3 = a 11 x 1 + a 12 x 2 + a 13 x 3 a 21 x 1 + a 22 x 2 + a 23 x 3 a 31 x 1 + a 32 x 2 + a 33 x 3 col 1 col 2 col 3 x 1 col 1 x 2 col 2 x 3 col 3 = x 1 a 11 a 21 a 31 + x 2 a 12 a 22 a 32 + x 3 a 13 a 23 a 33 Represented as a linear combination of column vectors
Matrix equation as a vector equation Now we have a vector equation, stating the same problem as Ax = b x 1 a 11 a 21 a 31 + x 2 a 12 a 22 a 32 + x 3 a 13 a 23 a 33 = b 1 b 2 b 3 So, we can represent any Ax = b equation as a vector equation. If b is in the span of column vectors, the system has either ONE or INFINTE solution. If b is NOT in the span of column vectors, the system has NO solution. But, we can find projection of b on the vector space, as best estimate.
Matrix equation (ONE solution) Example: y 2x + y = 5 4x 2y = 2 Row representation of above equation: 1 3 2 1 4 2 x y = 5 2 x
Vector equation (ONE solution) Span of col 1 Vector equation representation of previous example: dim 2 col 1 x 2 4 + y 1 2 = 5 2 col 2 b dim 1 b is in the span of col 1 and col 2. The only solution for this equation is x = 1, and y = 3 Span of col 2
Matrix equation (INFINITE solution) Example: 2x + y = 3 4x + 2y = 6 y Row representation of above equation: 2 1 x 4 2 y = 3 6 Both equations are the same line All points on the line are solutions x
Vector equation (INFINITE solution) Vector equation representation of previous example: x 2 4 + y 1 2 = 3 6 dim 2 b Span of col 1 and Span of col 2 col 1 and col 2 are linearly dependent b is in the span of col 1 and col 2 2x + y 1 2 = 3 6 Any combination of x and y that satisfies 2x + y = 3 is a solution. col 2 col 1 dim 1
Matrix equation (NO solution) Example: 2x + y = 3 4x + 2y = 4 y Row representation of above equation: 2 1 x 4 2 y = 3 4 The lines are parallel. There is no solution. x
Vector equation (NO solution) Vector equation representation of previous example: dim 2 Span of col 1 and Span of col 2 x 2 4 + y 1 2 = 3 4 col 1 and col 2 are linearly dependent col 2 col 1 b b is NOT in the span of col 1 and col 2 There is no solution dim 1
Vector equation (NO solution) dim 2 Projection of b Hence, in order to solve this we project b onto the span of col 1 and col 2 using Least Squares col 2 col 1 b dim 1