Abstract & Applied Linear Algebra (Chapters 1-2) James A. Bernhard University of Puget Sound

Abstract & Applied Linear Algebra (Chapters 1-2) James A. Bernhard University of Puget Sound

Copyright 2018 by James A. Bernhard

Contents 1 Vector spaces 3 1.1 Definitions and basic properties................. 4 1.2 Subspaces............................. 35 1.3 Spans................................ 45 1.4 Linearly independent sets.................... 56 2 Finite-dimensional vector spaces 65 2.1 Bases................................ 65 2.2 Dimension............................. 71 2.3 Echelon vectors.......................... 81 2.4 Coefficient nullspaces....................... 91 1

2 CONTENTS

Chapter 1 Vector spaces A real vector space is a set together with two operations called vector addition and scalar multiplication that satisfy certain properties. Elements of a real vector space are called vectors. Vector addition combines two vectors to produce a vector. Scalar multiplication combines a real number, called a scalar, and a vector to produce a vector. In this chapter we explore the basic aspects of vector spaces. 3

4 CHAPTER 1. VECTOR SPACES 1.1 Definitions and basic properties A real vector space is a set together with two operations called vector addition and scalar multiplication that satisfy certain properties. The set R n is the collection of finite sequences of length n, and it has a standard vector addition and scalar multiplication defined on it. This set and these operations form the prototypical real vector space. Basic theory Linear algebra is the study of vector spaces and linear transformations. We begin by introducing vector spaces, and later in this text we will introduce linear transformations. In order to understand real vector spaces, you need to understand some basic terminology about sets. Although a rigorous definition of a mathematical set is beyond the scope of this text, we give the following intuitive definition, along with some related notation. Definition 1.1 A set is a collection of mathematical entities, which are called members or elements of the set. To denote that an entity x is a member of a set X, we write x X. To denote that X is the set consisting of the mathematical entities x 1, x 2,..., x n, we write X = {x 1, x 2,..., x n }. To denote that X is the set of all such-and-such satisfying certain conditions, we write X = {such-and-such certain conditions}. A set is called finite if it has only finitely many members. The number of elements in a finite set X is denoted by X. The notation x X is read x is in X or x is an element of X.

1.1. DEFINITIONS AND BASIC PROPERTIES 5 Another useful concept related to sets is that of containment, which we now define. Definition 1.2 Let X 1, X 2 be sets. Then X 1 X 2 if every element of X 1 is also an element of X 2. This relationship can be stated in various ways, such as: X 1 is contained in X 2. X 1 is a subset of X 2. X 2 contains X 1. Two sets X 1, X 2 are called equal if any of the following three equivalent conditions is satsified: (i) X 1 and X 2 contain exactly the same elements. (ii) An entity x is an element of X 1 if and only if it is an element of X 2. (iii) X 1 X 2 and X 2 X 1. We most often use the third of the conditions for equality when we prove that two sets are equal. One further concept related to sets will be useful for our purposes here. Definition 1.3 Let X 1, X 2 be sets. Then the Cartesian product of X 1 with X 2, denoted by X 1 X 2, is the set of pairs of elements, the first of which is in X 1 and the second of which is in X 2. The order of elements in the pairs that are members of a Cartesian product matters, so X 1 X 2 is not the same as X 2 X 1. The set X 1 X 2 consists of pairs whose first element is in X 1 and whose second element is in X 2, while the set X 2 X 1 consists of pairs whose first element is in X 2 and whose second element is in X 1. We often use Cartesian products to describe operations. For example,

6 CHAPTER 1. VECTOR SPACES we can think of the operation of subtraction of real numbers as a function from R R to R because it uses two real numbers as input and gives a real number as output. Also, in subtraction the order of the two real numbers matters: for example, 3 2 is not the same as 2 3. The order doesn t matter for addition of real numbers, but we can still view addition as a function from R R to R with an extra property that while interchanging the numbers gives a different expression (such as 3 + 2 instead of 2 + 3), the resulting sum is the same. That property is, however, beyond what the Cartesian product itself guarantees. These various terms related to sets are used throughout higher mathematics, so you should become familiar with them if you aren t already. We next introduce R n, the set that we will study most in depth in this text, and some basic terminology related to it.

1.1. DEFINITIONS AND BASIC PROPERTIES 7 Definition 1.4 A real column vector is a sequence of finitely many real numbers. These real numbers are called entries in the real column vector. The real column vector whose entries (in order) are v 1, v 2,..., v n is denoted by v 1 v 2.. v n For any positive integer n, the set of all real column vectors with exactly n entries is denoted by R n, so: v 1 v 2 R n = {. v 1, v 2,..., v n R}. v n v 1 v 2 v n w 1 w 2 Two real column vectors.,. Rn are equal if w n v 1 = w 1, v 2 = w 2,..., and v n = w n. We write the names of real column vectors as lowercase letters with arrows over them, as in v 1 v 2 v =.. v n The notation R n is read by pronouncing both symbols: r n. From elementary geometery, you are no doubt familiar with how elements of R 2 can be used to specify points in a plane. In linear algebra, we still use this method, but we also have two other interpretations for elements of R 2 : as directed line segments at the origin and as translat-

8 CHAPTER 1. VECTOR SPACES able directed line segments. To clarify these interpretations, a directed line segment is a line segment that has a specified direction from one of its endpoints to the other. A directed line segment is usually drawn as an arrow whose tail is at the segment s starting point and whose head is at the segment s ending point. Also, to translate a shape is to relocate the shape in such a way that every point in the shape is moved by exactly the same amount and in the same direction. A directed line segment is translatable if it is allowed to be translated. This gives us three ways to interpret ordered pairs of real numbers in linear algebra, as pictured in Figure 1.5. as a translatable directed line segment as a point as a directed line segment at the origin Figure 1.5: Three ways to view [ ] 1 R 2 2. Each of these interpretations has its advantages and disadvantages, and we will shift freely among the interpretations in order to suit the problem at hand. Also, all three interpretations carry over readily from the plane to 3-dimensional space, and the associated picture is similar. Even more interestingly, although these geometric interpretations are harder to depict when n > 3, they extend to general n-tuples of real numbers by analogy. And as we will see, they actually extend even further than that too, so by building a strong visual and geometric understanding of ordered pairs and triples in linear algebra, you will be building a geometric intuition that carries over into a wide range of settings throughout linear algebra. Continuing with our study of R n, the following column vectors are noteworthy enough that they have their own name.

1.1. DEFINITIONS AND BASIC PROPERTIES 9 Definition 1.6 For i {1,..., n}, the i-th standard basis vector in R n is the column vector e i R n whose i-th entry equals 1 and whose other entries are all 0, meaning that: 1 0 0 e 1 = 0., e 2 = 1.,,..., e n = 0.. 0 0 1 For any specific n, the set of all these vectors is called the standard basis E for R n : E = { e 1, e 2,..., e n }. We will define the term basis (plural: bases, pronounced base ease ) in Section 2.1, but for now just understand that a basis is a set of vectors that satisfies certain conditions. The vectors in a basis are called basis vectors. The term standard is used for this particular basis because it is the most commonly used basis in R n. Most other real vector spaces do not have a standard basis. Note that the standard basis for R n is different for each n. For example, [ ] 1 1 the vector e 1 = in R 0 2 is different from the vector e 1 = 0 in R 3, and 0 so on. This means that when we refer to standard basis vectors, we must specify which R n they belong to. The letter E is a script version of E. In this text, we will often use script letters to denote finite sets of vectors. The standard basis vectors for R 2 and R 3 are shown as directed line segments at the origin in the Figure 1.7. As is apparent, these line segments are each 1 grid unit long, and they point in the positive direction along the coordinate axes.

10 CHAPTER 1. VECTOR SPACES e 2 e 1 Figure 1.7: The standard basis vectors in R 2. In our usual view of R 2, the standard basis vectors are ordered counterclockwise: e 2 points in a direction 90 degrees counterclockwise from e 1. There is no standard position from which to view the standard basis vectors in R 3, but they are ordered by what is known as the right-hand rule: if you point your right forearm in the approximate direction of e 1 and curl the fingers on your right hand in the approximate direction of e 2, then your right thumb will point in the approximate direction of e 3. If you were to try the same thing with your left hand, your left thumb would point in the opposite direction to e 3. Now that we are familiar with some aspects of R n as a set, we can turn our attention to things that we can do with elements of R n. As we explain next, we can add them to each other and multiply them by real numbers.

1.1. DEFINITIONS AND BASIC PROPERTIES 11 Definition 1.8 Let n be a positive integer. The standard vector addition on R n is defined by: x 1 y 1 x 1 + y 1 x 2. + y 2. = x 2 + y 2., x n y n x n + y n x 1 x 2 x n y 1 y 2 y n for all.,. Rn. In words, this definition stipulates that vectors in R n add entrywise. We refer to this vector addition as standard because although there are other ways to add n-tuples of real numbers, this is by far the most common one. As examples of the standard vector addition in R n : [ ] 2 + 1 [ ] 1 = 3 [ ] 1 4 and 3 2 5 1 + 1 = 0. 2 1 1 The standard vector addition in R 2 can be depicted geometrically most readily if we think of vectors as translatable directed line segments. From this point of view, if we place v 1 with its tail at the origin and v 2 with its tail at the head of v 1, then v 1 + v 2 extends from the origin to the tail of v 2, as pictured in Figure 1.9.

12 CHAPTER 1. VECTOR SPACES v 1 + v 2 v 2 v 2 v 1 + v 2 = v 2 + v 1 v 1 v 1 Figure 1.9: Vector addition. Figure 1.10: Commutativity of vector addition. The same type of picture can be drawn in R 3 and imagined by analogy in R n when n > 3. To help with this analogy, notice by drawing some examples that even in R 3, vectors v 1, v 2, and v 1 + v 2 all lie within a plane. This plane is essentially a copy of R 2, so the extra directions that are possible in R 3 don t really change the picture for vector addition. They only change the viewing position for the picture. By analogy, think of the same phenomenon as holding in R n in general. An operation is called commutative if which argument is first and which is second does not affect the result of the operation. For addition in R n, this would mean that: v 1 + v 2 = v 2 + v 1 for all vectors v 1, v 2 R n. Using the commutativity of the addition of real numbers, we can prove directly from the definition that the standard vector addition in R n is in fact commutative, but this can also be seen geometrically in R 2. Arriving at the upper right corner of the parallelogram via the lower sides corresponds to the sum v 1 + v 2, whereas arriving at the upper right corner of the parallelogram via the upper sides corresponds to the sum v 2 + v 1. This geometric demonstration of the commutativity of vector addition is illustrated in Figure 1.10.The same type of picture can be drawn in R 3 and imagined by analogy in R n for n > 3. Now that we know how to add vectors in R n, we proceed to define how to multiply a vector in R n by a real number.

1.1. DEFINITIONS AND BASIC PROPERTIES 13 Definition 1.11 Let n be a positive integer. The standard scalar multiplication in R n is defined for any a R and. Rn by: v 1 av 1 v 2 a. = av 2. v n av n v 1 v 2 v n As with standard vector addition, the term standard here refers to the fact that although there are many other types of scalar multiplication that can be defined in R n, this definition is by far the most common. As examples of standard scalar multiplication in R 2 and R 3, we have: [ [ 1 3 3 = 2] 6] and 1 3 3/2 1 = 1/2. 2 2 1 As pictured in Figure 1.12, the standard scalar multiplication in R 2 can be represented geometrically if we think of vectors as directed line segments at the origin. Using this approach, suppose that v R 2 and a R. Then the vector a v is the vector v with each coordinate scaled by a factor of a (hence the term scalar multiplication). This means that a is the factor by which the vector is stretched, and the positivity or negativity of a determines whether the direction of the vector gets reversed: it stays the same for positive a and reverses for negative a.

14 CHAPTER 1. VECTOR SPACES 1.8 v 1 2 v v v Figure 1.12: Scalar multiplication in R 2. These considerations lead to several possibilities. If a > 1, the vector v will be stretched to obtain a v. If a = 1, then a v = v. If 0 a < 1, the vector v will be shrunk to obtain a v, all the way to a point if a = 0. For negative values of a, the direction of v will be reversed and v will also be stretched or shrunk (or neither) according to the value of a. Examples of these cases are illustrated in Figure 1.12. The same type of picture can be drawn in R 3 as well and imagined by analogy in other R n. The set R n with these standard operations forms the prototype that all real vector spaces are patterned after, and it is the primary example to consider whenever you work with real vector spaces. With this in mind, we now define real vector spaces in general.

1.1. DEFINITIONS AND BASIC PROPERTIES 15 Definition 1.13 A real vector space is a set V with two operations, known as vector addition V V V and scalar multiplication R V V, satisfying the following properties: 1. Vector addition is associative: (v 1 + v 2 ) + v 3 = v 1 + (v 2 + v 3 ) for all v 1, v 2, v 3 V. 2. Vector addition is commutative: v 1 + v 2 = v 2 + v 1 for all v 1, v 2 V. 3. There exists a zero vector (or additive identity) 0 V satisfying 0 + v = v for all v V. 4. For each v V, there exists an additive inverse v inv V satisfying v + v inv = 0. 5. Scalar multiplication is associative, meaning: a(bv) = (ab)v for all a, b R and v V. 6. The real number 1 R is a scalar multiplicative identity, in the sense that 1 v = v for all v V. 7. Scalar multiplication distributes over addition in R: (a + b)v = av + bv for all a, b R and v V. 8. Scalar multiplication distributes over vector addition: a(v 1 + v 2 ) = av 1 + av 2 for all a R and v 1, v 2 V. An element of a real vector space is called a vector, and in the context of a real vector space, an element of R is called a scalar.

16 CHAPTER 1. VECTOR SPACES There is no standard notation for vectors in general. Some vector spaces have a familiar notation, and we ordinarily use that. For example, in R n we will write vectors with arrows over them, as in v. When we form vector spaces from polynomials in x, we will write vectors with notation such as p(x). For vector spaces formed from other types of functions, we may use notation such as f or f (x) for vectors. In this text, we will use boldface lowercase letters to denote vectors in real vector spaces that don t already have their own notation for their members, as in v V. Vector spaces are the central objects of study in linear algebra, and in this text we will focus primarily on real vector spaces. Vector spaces can have other types of scalars besides real numbers, such as complex numbers or rational numbers, but we will work primarily with real vector spaces before exploring such generalizations. Everything that one can prove about real vector spaces is deduced from this definition, so you should memorize it. When we prove some initial results about real vector spaces, we will usually cite this definition directly. After we have used the definition to build up some of the basic properties of real vector spaces, then we will often cite those properties rather than citing the definition directly, but the definition is still behind it all. So far we have asserted that R n is an example of a real vector space. We now state this formally as a proposition. Proposition 1.14 The set R n with the standard operations of vector addition (Definition 1.8) and scalar multiplication (Definition 1.11) is a real vector space whose zero vector 0 is the vector 0 whose entries are all zeroes, namely 0 0 = 0. Rn. 0 Proof. See Example Problem 1.32 on page 28. When we refer to R n from now on, unless we specify otherwise we mean the real vector space R n with its standard operations. For example, when we state that a particular set is a subspace of R n, we mean that it is

1.1. DEFINITIONS AND BASIC PROPERTIES 17 a subspace of the real vector space R n with its usual operations. We can also define another vector space, that of polynomials of degree at most n. Definition 1.15 For any nonnegative integer n, we denote the space of all polynomials in a single variable x with real coefficients of degree at most n by P n, so: P n = {a 0 + a 1 x + + a n x n a 0,..., a n R}. This set has its own standard operations of addition and scalar multiplication, defined by: (a 0 + a 1 x + + a n x n )+(b 0 + b 1 x + + b n x n ) = (a 0 + b 0 ) + (a 1 + b 1 )x + + (a n + b n )x n, and c(a 0 + a 1 x + + a n x n ) = ca 0 + ca 1 x + + ca n x n for all a 0,..., a n, b 0,..., b n, c R. This set with these operations forms a vector space, as we now state formally in the following proposition. Proposition 1.16 The set P n with its standard operations of vector addition and scalar multiplication is a real vector space whose zero vector 0 is the polynomial z(x) = 0 + 0x + + 0x n P n. Proof. We leave this as an exercise for the reader. Further results The definition of a real vector space has two immediate and important consequences, given in the following two propositions.

18 CHAPTER 1. VECTOR SPACES Proposition 1.17 Let V be a real vector space. Then the zero vector in V is unique. Proof. See Example Problem 1.29 on page 25. With this proposition, we can safely refer to the zero vector in a vector space, even though the definition of a real vector space (Definition 1.13) refers only to a zero vector. Unless otherwise specified, from now on we denote the zero vector in a vector space by 0. Having established that the zero vector in a vector space is unique, we now turn our attention to additive inverses. Proposition 1.18 Let V be a real vector space, and let v V. Then the additive inverse of v is unique. Proof. See Example Problem 1.30 on page 26. With this proposition, we can safely refer to the additive inverse of a vector, even though the definition of a real vector space (Definition 1.13 on page 15) refers only to an additive inverse. In light of this result, we introduce some familiar notation. Definition 1.19 Let V be a real vector space, and let v V. We denote the unique additive inverse of v by v. Because additive inverses are unique, we can define subtraction of vectors. Definition 1.20 Let V be a real vector space, let v 1, v 2 V. We define the difference v 1 v 2 by: v 1 v 2 = v 1 + ( v 2 ). As usual, we refer to this operation as subtracting v 2 from v 1.

1.1. DEFINITIONS AND BASIC PROPERTIES 19 Note that subtraction in a real vector space is defined in terms of addition and additive inverses. It is not part of the definition of a vector space. In order to define it, we first needed to make sure that additive inverses were unique. To understand subtraction geometrically, we look at it in R 2. The vector v 1 v 2 is the vector that must be added to v 2 to obtain v 1. As the following picture suggests, the vector v 1 v 2 can be remembered as tip minus tail: if we think of v 1 v 2 as a translatable directed line segment, the tip of v 1 v 2 is at v 1, and its tail is at v 2. This is shown in Figure 1.21. v 1 v 2 v 1 v 2 Figure 1.21: Vector subtraction in R 2. The next two propositions give us some useful basic results about real vector spaces. Proposition 1.22 Let V be a real vector space, and let v V. Then 0v = 0. Proof. First, note that 0 = 0 + 0, by a familiar property of the real number 0. Scalar multiplying v by both sides of this equation gives: 0v = (0 + 0)v.

20 CHAPTER 1. VECTOR SPACES Using the distributivity of scalar multiplication over addition in R, given in the definition of a real vector space (Definition 1.13), on the right side of this equation, we have: 0v = 0v + 0v. If we add (0v) to both sides of this equation, we have: 0v + ( (0v)) = (0v + 0v) + ( (0v)). Since vector addition is associative by the definition of a real vector space, this gives us: (0)v + ( (0v)) = 0v + (0v + ( (0v))). Using the defining property of additive inverses, this simplifies to: 0 = 0v + 0. By the defining property for the zero vector applied to the right side of the equation, this means that: 0 = 0v, which completes the proof of the proposition. Proposition 1.23 Let V be a real vector space, and let a R. Then a0 = 0. Proof. See Example Problem 1.31 on page 27. Proposition 1.24 Let V be a real vector space, and let v V. Then 1v = v. Proof. To show that 1v is the additive inverse of v, we must show that it satisfies the defining property of the additive inverse of v given in the definition of a real vector space (Definition 1.13 on page 15), namely that: v + ( 1v) = 0.

1.1. DEFINITIONS AND BASIC PROPERTIES 21 To show this, we compute directly, using the properties given in the definition of a real vector space (Definition 1.13 on page 15). Since 1 is the scalar multiplicative identity in a real vector space: v + ( 1v) = 1v + ( 1v). Since scalar multiplication distributes over addition in R, the right side can be simplified to yield: Since 1 + ( 1) = 0, this gives us: = (1 + ( 1))v. = 0v. Since 0 times any vector equals the zero vector (by Proposition 1.22 on page 19), the right side of this equation equals the zero vector 0, so 1v is the additive inverse of v. Now that we are familiar with the basic properties of vector spaces, we can extend them with the following useful proposition. Proposition 1.25 Let V a real vector space. Addition of any finite number of vectors can be performed in any order, meaning that parentheses can be placed as desired and the left to right position of the vectors can be rearranged as desired. Similarly, any finite number of scalar multiplications can be performed in any order. Proof. These follow directly from the definition of a real vector space (Definition 1.13), but proving them would here would sidetrack us greatly and would not be particularly informative, so we will not do so here. The interested reader may prove this proposition as an exercise, but at this point it is not recommended. Example Exercises

22 CHAPTER 1. VECTOR SPACES Example Exercise 1.26 Solve for x R in the equation [ ] [ ] x 2 x = 2 x 2. + 3x Solution. By the definition of equals in R n (Definition 1.4 on page 7) two elements of R 2 are equal if and only if their two corresponding entries are equal. This gives us two equations, one for each entry: x 2 = x 2 = x 2 + 3x. Both of these equations must hold in order for the original vector equation to hold. The first equation yields x 2 + x = 0, so x(x + 1) = 0, meaning that x = 0 or x = 1. The second equation gives us that x 2 + 3x + 2 = 0, so (x + 2)(x + 1) = 0, which means that x = 1 or 2. Since the equation in R 2 is true if and only if the individual entry equations are both true, x = 1 is the only solution to the original equation in R 2. Example Exercise 1.27 Let W R 3 be defined as follows: x W = { 0 R 3 }. 0 With the usual operations of vector addition and scalar multiplication in R 3, is W a real vector space? Solution. Yes, W is a vector space with these operations. To see this, we first check that the usual operations in R 3 do define operations on W. Then we check that these operations satisfy the defining conditions for a real vector space (Properties 1 through 8 of Definition 1.13 on page 15).

1.1. DEFINITIONS AND BASIC PROPERTIES 23 To show that the usual vector addition in R 3 restricted to W defines an operation on W, let v, w W. By the definition of W, v w v = 0 and w = 0 0 0 for some v, w R. Then v w v + w v + w = 0 + 0 = 0, 0 0 0 which is in W by the definition of W. Since v, w W were arbitrary, this holds for all v, w W, so restricting the usual vector addition in R 3 to W defines an operation W W W. To show that the usual scalar multiplication in R 3 restricted to W defines an operation on W, let a R and v W. By the definition of W, for some v R. Then v v = 0 0 v av a v = a 0 = 0, 0 0 which is in W by the definition of W. Since a R and v W were arbitrary, this holds for all a R and v W, so restricting the usual scalar multiplication in R 3 to W defines an operation R W W. 0 Also, by the definition of W, 0 W. Since 0 0 0 + v = v 0

24 CHAPTER 1. VECTOR SPACES for all v R 3, then the same holds for all v W R 3. This means that there is a zero vector in W, and it is the same zero vector as in R 3, so Property 3 is satisfied. Vector addition in W is associative and commuative, since it is associative an commutative for all vectors in R 3 and W R 3, so Properties 1 and 2 are satisfied. We have shown that the inverse of a vector is 1 times that vector, and we have shown that a scalar times a vector in W is in W. This means that each vector in W has an additive inverse in W, so Property 4 is satisfied. Associativity of scalar multiplication, the scalar 1 acting as a multiplicative identity, scalar multiplication distributing over addition in R, and scalar multiplication distributing over vector addition all hold for all vectors in R 3, so they hold in particular for all vectors in W R 3. This means that Properties 5 through 8 are satisfied, so W is a real vector space. Example Exercise 1.28 Let W R 3 be defined as follows: x W = { y R 3 z = x 2 + y 2 }. z With the usual operations of vector addition and scalar multiplication in R 3, is W a real vector space? Solution. No, W is not a real vector space with these operations. In particular, the usual scalar multiplication in R 3 restricted to W is not an operation R W W. To see this, note that by the definition of W, 1 1 W 2 because 2 = 1 2 + 1 2. However, also by the definition of W, 1 2 2 1 = 2 W 2 4

1.1. DEFINITIONS AND BASIC PROPERTIES 25 because 4 = 2 2 + 2 2. This proves that the usual scalar multiplication in R 3 restricted to W is not an operation R W W, so by the definition of a real vector space (Definition 1.13 on page 15), W is not a real vector space under the usual operations on R 3. Remark To prove that W is not a real vector space under the given operations, we first had to realize that it wasn t. For that, we examined the characteristics of a real vector space in the definition of a real vector space (Definition 1.13 on page 15). When we found one that didn t hold (in this exercise, the specified scalar multiplication was not an operation on R W W), we used that to prove the result. Note that since all parts of the definition must hold for W to be a vector space, we only had to show that one part of the definition did not hold in order to show that W is not a vector space under these operations. Example Problems Example Problem 1.29 Let V be a real vector space. Prove that the zero vector in V is unique. Solution. Let z 1 and z 2 be zero vectors for V. Since z 1 is a zero vector in V, by the definition of a zero vector given in the defintion of a real vector space (Definition 1.13 on page 15) we have: z 2 = z 1 + z 2. By the commutativity of vector addition, which is also part of the definition of a real vector space, we can rearrange the order of the two terms on the right to obtain: z 2 = z 2 + z 1 Because z 2 is a zero vector in V, this gives us, by the definition of a zero vector, which is part of the definition of a real vector space: which completes the proof. z 2 = z 1,

26 CHAPTER 1. VECTOR SPACES Remark This illustrates the typical technique for proving that something is unique: assume that you have two of them and then show that those two must in fact be equal. Example Problem 1.30 Let V be a real vector space, and let v V. Then the additive inverse of v is unique. Solution. Assume that w 1, w 2 V are both additive inverses of v. By the defining property of the zero vector given in the definition of a real vector space (Definition 1.13 on page 15), we have: w 1 = 0 + w 1. Because w 2 is an additive inverse of v, we can substitute w 2 + v = 0 into the right side of this equation to obtain: w 1 = (w 2 + v) + w 1. By the associativity of vector addition in the definition of a real vector space, this means: w 1 = w 2 + (v + w 1 ). By the commutativity of vector addition in the definition of a real vector space, this gives us: w 1 = w 2 + (w 1 + v). Because w 1 is an additive inverse of v, this gives by the definition of an additive inverse in the definition of a real vector space: w 1 = w 2 + 0. By the defining property of the zero vector in the definition of a real vector space, this yields: which completes the proof. w 1 = w 2,

1.1. DEFINITIONS AND BASIC PROPERTIES 27 Remark This again illustrates the typical technique for proving that something is unique: assume that you have two of them and then show that those two must in fact be equal. Example Problem 1.31 Let V be a real vector space, and let a R. Then a0 = 0. Solution. If a = 0, then a0 = 0 because 0 times any vector is 0 (by Proposition 1.22 on page 19). If a = 0, then to show that a0 satisfies the defining characteristic of 0 given in the definition of a real vector space (Definition 1.13 on page 15), let v V. Since a = 0, we can multiply by 1/a, so: a0 + v = a(0 + (1/a)v) = a((1/a)v) = 1v = v. Since a0 satisfies the defining characteristic of 0, then a0 = 0 by the uniqueness of the zero vector (Proposition 1.17 on page 18). Remark In higher mathematics, an object can be defined by the properties that it satisfies. This example shows how to prove that something is such an object: you show that the given thing does in fact satisfy the defining properties of the object. This is a very general method that is used widely in higher mathematics. In this particular example, we want to show that something (namely a0) is a zero vector. In a real vector space, a zero vector is defined as any vector 0 that satisfies 0 + v = v for all v V (and we later showed that only one vector has this property). So to prove that a0 is the zero vector, we show that a0 does indeed satisfy 0 + v = v for all v V. This problem also illustrates how to prove something is true for all objects of a certain type: you declare an arbitrary object of that type ( Let v V in this problem) and then show that the result holds for that declared object ( 0 + v = v for this particular declared v). Be-

28 CHAPTER 1. VECTOR SPACES cause the declared object is arbitrary, meaning that you know nothing about it except that it is the specified type of object, then the result holds for all objects of that type. Example Problem 1.32 Show that with the usual operations of scalar multiplication, R n is a real vector space for all positive integers n. Solution. Let n be a positive integer. We now check each of the items in the definition of a real vector space (Definition 1.13 on page 15). For this, let a, b R and v, w, x R n with: v 1 v 2 v n w 1 w 2 w n x 1 x 2 v =., w =., x =. Throughout our computations to verify the various items in the definition, we will use the definitions of standard vector addition in R n (Definition 1.8 on page 11) and standard scalar multiplication in R n (Definition 1.11 on page 13). 1. To check associativity of vector addition, we compute that: v 1 v 2 v n w 1 w 2 w n x n x 1 x 2 ( v + w) + x = (. +. ) +. v 1 + w 1 x 1 v 2 + w 2 =. + x 2. v n + w n x n (v 1 + w 1 ) + x 1 (v 2 + w 2 ) + x 2 =. (v n + w n ) + x n x n

1.1. DEFINITIONS AND BASIC PROPERTIES 29 By the associtivity of addition in R, we can rearrange the parentheses in each entry to obtain: v 1 + (w 1 + x 1 ) v 2 + (w 2 + x 2 ) =. v n + (w n + x n ) v 1 w 1 + x 1 v 2 =. + w 2 + x 2. v n w n + x n v 1 v 2 v n w 1 w 2 w n x 1 x 2 x n =. + (. +. ) = v + ( w + x). 2. To check commutativity of vector addition, we compute that: v 1 v 2 v n w 1 w 2 w n v + w =. +. v 1 + w 1 v 2 + w 2 =. v n + w n

30 CHAPTER 1. VECTOR SPACES Because addition in R is commutative, we can switch the order of addition in each entry to obtain: w 1 + v 1 w 2 + v 2 =. w n + v n w 1 w 2 w n v 1 v 2 =. +. = w + v. 3. To show that the vector in R n with all zero entries is the zero vector under the usual operations, we compute that: 0 0 v 1 0 + v 1 v 1 0. + v = 0. + v 2. = 0 + v 2. = v 2. = v. 0 0 v n 0 + v n v n Since v (selected above) was arbitrary, this equation holds for all v R n. This means that the vector in R n with all zero entries satisfies the defining property for the zero vector, so it is the zero vector in R n. 4. To show that every element of R n has an additive inverse, we compute that: v 1 v 1 v 1 v 1 + ( v 1 ) 0 v 2 v +. = v 2. + v 2. = v 2 + ( v 2 ). = 0. = 0. v n v n v n v n + ( v n ) 0 v 1 v 2 This means that satisfies the defining property for the additive inverse of v, so it is the additive inverse of v. Since v (selected. v n above) was arbitrary, every vector in R n has an additive inverse. v n

1.1. DEFINITIONS AND BASIC PROPERTIES 31 5. To check the associativity of scalar multiplication, we compute that: v 1 v 2 v n a(b v) = a(b. ) bv 1 bv 2 = a. bv n a(bv 1 ) a(bv 2 ) =. a(bv n ) Because multiplication in R is associative, we can rearrange the parentheses in each entry to obtain: (ab)v 1 (ab)v 2 =. (ab)v n v 1 v 2 = (ab). = (ab) v. v n Since a, b, and v (selected above) were arbitrary, this holds for all a, b R and all v R n.

32 CHAPTER 1. VECTOR SPACES 6. To check that 1 times any vector equals that vector, we compute that: v 1 (1)(v 1 ) v 1 v 2 1 v = 1. = (1)(v 2 ). = v 2. = v. (1)(v n ) v n v n Since v (selected above) was arbitrary, this holds for all v R n. 7. To check that scalar multiplication distributes over addition in R, we compute that: v 1 v 2 v n (a + b) v = (a + b). (a + b)(v 1 ) (a + b)(v 2 ) =. (a + b)(v n )

1.1. DEFINITIONS AND BASIC PROPERTIES 33 Because multiplication in R distributes over addition in R, we can multiply each entry out to obtain: av 1 + bv 1 av 2 + bv 2 =. av n + bv n av 1 bv 1 av 2 =. + bv 2. av n bv n v 1 v 2 v n v 1 v 2 = a. + b. = a v + b v. v n Since a, b, and v (selected above) were arbitrary, this holds for all a, b R and all v R n. 8. To check that scalar multiplication distributes over vector addition,

34 CHAPTER 1. VECTOR SPACES we compute that: v 1 v 2 v n w 1 w 2 a( v + w) = a(. +. ) w n v 1 + w 1 v 2 + w 2 = a. v n + w n a(v 1 + w 1 ) a(v 2 + w 2 ) =. a(v n + w n ) Because multiplication in R distributes over addition in R, we can multiply each entry out to obtain: av 1 + aw 1 av 2 + aw 2 =. av n + aw n v 1 v 2 v n w 1 w 2 w n = a. + a. = a v + a w. Since a, v, and w (selected above) were arbitrary, this holds for all a R and all v, w R n. Since we have verified all eight items in the definition of a vector space, we have shown that R n is a vector space. In the process, we have also shown that the zero vector in R n is the vector whose entries are all zeroes.

1.2. SUBSPACES 35 1.2 Subspaces A subspace is a subset of a real vector space that is itself a real vector space under the same operations as the original real vector space. Another way to characterize a subspace is as a subset of a real vector space that contains 0, is closed under vector addition, and is closed under scalar multiplication. Basic theory Having established the definition and basic characteristics of real vector spaces, we now turn to our next topic: subspaces, or vector spaces within vector spaces. In order to avoid getting bogged down in basic computations we will no longer cite every application of the properties in the definition of a real vector space (Definition 1.13 on page 15) as we have up to this point. Rather, we will cite then only when they aid in making a proof more clear. Also, we will use the extended version of associativity (stated in Proposition 1.25 on page 21) freely and without citation. From Euclidean geometry, we are already familiar with two types of subsets that generalize readily from R 2 and R 3 to other vector spaces: lines and planes. In linear algebra, these generalize to the idea of subspaces, which we now define. Definition 1.33 Let V be a real vector space. A subspace of V is a subset W V that is itself a real vector space under the vector addition and scalar multiplication from V. Note that a subspace must use the same operations as the full space. If a vector space W is a subset of a vector space V, it does not necessarily follow that W is a subspace of V. Example 1.39 on page 39 shows that the entire space V is a subspace of V. Any subspace that is not the entire subspace is called a proper subspace. At this point, we might wonder whether a subspace has to have the same zero vector as the real vector space that it is contained in. In fact it

36 CHAPTER 1. VECTOR SPACES must, as the following proposition tells us. Proposition 1.34 Let V be a real vector space with zero vector 0 V, and let W V be a subspace. Then: 1. 0 V W 2. 0 V is the zero vector of W. Proof. To prove the first statement, note that because W is a vector space, it must have a zero vector 0 W W. Since 0 W W and W V, then 0 V. Since zero times any vector is the zero vector (by Proposition 1.22 on page 19), then 0 0 W = 0 V. Because W is a real vector space under this operation of scalar multiplication, then 0 0 W W, so 0 V W. To prove the second statement, note that we have already shown that 0 0 W = 0 V. However, because W is a real vector space and because zero times any vector is the zero vector in that vector space, then 0 0 W = 0 W. Putting these two equations together, we have that 0 V = 0 W. In short, this proposition tells us that the zero vector of a real vector space is also the zero vector of all its subspaces. We don t have to figure out a new zero vector for each subspace. We now prove the following useful alternative characterization of subspaces. Proposition 1.35 Let V be a real vector space with zero vector 0 V, and let W V. Then W is a subspace if and only if all of the following three conditions hold: 1. 0 W, 2. W is closed under vector addition: v + w W for all v, w W, 3. W is closed under scalar multiplication: av W for all a R and all v W.

1.2. SUBSPACES 37 Proof. For the forwards implication, assume that W is a subspace. Since the zero vector of a real vector space is contained in all its subspaces (by Proposition 1.34 on page 36), then 0 W. Since vector addition on a real vector space W is a function W W W by the definition of a real vector space, then W is closed under its vector addition, which is the same as the vector addition in V. Also, since scalar multiplication on a real vector space W is a function R W W, then W is closed under its scalar multiplication, which is the same as the scalar multiplication in V. This proves the forwards implication. For the backwards implication, assume that W satisfies the three properties listed in the statement of the proposition. By closure under vector addition, vector addition in V defines a function W W W when its domain is restricted from V V to W W. Also, by closure under scalar multiplication, scalar multiplication in V defines a function R W W when its domain is restricted to W. With these definitions of vector addition and scalar multiplication, we verify the defining conditions of a vector space: 1. Vector addition is associative because it is associative for all vectors in V, including those in W. 2. Vector addition is commutative because it is commutative for all vectors in V, including those in W. 3. Since the zero vector of a vector space is contained in all subspaces and is the zero vector for each subspace (by Proposition 1.34 on page 36), then W has a zero vector for its operations, namely the zero vector for V. 4. Let w W. Since additive inverses are obtained by scalar multiplication by 1 (by Proposition 1.24 on page 20), the additive inverse of w is 1w. Since W is closed under scalar multiplication, then w = 1w W, so additive inverses exist in W. 5. Associativity of scalar multiplication holds for all vectors in W because it holds with the same scalar multiplication for all vectors in V.

38 CHAPTER 1. VECTOR SPACES 6. The property 1w = w holds for all w W because it holds for all w V. 7. Scalar multiplication distributes over addition in R for all vectors in W because it does so for all vectors in V. 8. Scalar multiplication distributes over vector addition for all vectors in W because it does so for all vectors in V. Since all the conditions for a vector space are satisfied, then W is a vector space with the same operations as V, which completes the proof of the proposition. Further results We now give three propositions that provide further examples of subspaces. Proposition 1.36 Let W R 2 be a line. Then W is a subspace if and only if W contains the origin. Proof. See Example Problem 1.43 on page 42. Proposition 1.37 Let W R 3 be a line. Then W is a subspace if and only if W contains the origin. Proof. Exercise for the reader. Proposition 1.38 Let W R 3 be a plane. Then W is a subspace if and only if W contains the origin.

1.2. SUBSPACES 39 Proof. Exercise for the reader. With our earlier results and these propositions, we now have that all of the following are subspaces of R 2 : {0}. any line containing the origin. R 2 itself. We have also learned that all of the following are subspaces of R 3 : {0}. any line containing the origin. any plane containing the origin. R 3 itself. Make sure that you know the subspaces of R 2 and R 3 in these lists, since these are the subspaces that should come to mind first whenever you are trying to understand anything about subspaces. Example Exercises We now give examples of two subspaces that are found in every real vector space. Example Exercise 1.39 Prove that for any real vector space V, the set V is a subspace of itself. Solution. Since V V and V is a real vector space under its own vector space operations, then V is a subspace of V by the definition of a subspace (Definition 1.33 on page 35). Example Exercise 1.40 Prove that in any real vector space V, the set {0} is a subspace.

40 CHAPTER 1. VECTOR SPACES Solution. We have that: 1. 0 {0} by the definition of {0}. 2. {0} is closed under addition because 0 + 0 = 0 by the definition of the zero vector of a real vector space (Definition 1.13 on page 15). 3. {0} is closed under scalar mutiplication because any scalar times 0 equals 0 (Proposition 1.22 on page 19). Since these three properties characterize subspaces (by Proposition 1.35), this means that {0} is a subspace. Remark The subspace {0} is called the zero subspace; a subspace that is not the zero subspace is called a nonzero subspace. Example Exercise 1.41 Let W be the subset of R 2 defined by: [ ] x1 W = { R 2 x 2 = x 3 1 }. Is W a subspace of R 2? x 2 Solution. No, [ W ] is not a subspace of R 2. To see this, note that by the defninition of W, W because 1 = 1 1 1 3. However, also by the definition [ ] [ ] 1 2 of W, 2 = W because 2 = 2 1 2 3. This shows that W is not closed under scalar multiplication. Since subspaces are closed under scalar multiplication (by Proposition 1.35 on page 36), this proves that W is not a subspace. Remark Proposition 1.35 on page 36 asserts that W is a subspace if and only if it satisfies three conditions. This means that to show that W is not a subspace, then we need only show that one of the conditions does not hold. Since closure under vector addition and under scalar

1.2. SUBSPACES 41 multiplication assert something about all vectors in W and all scalars, the usual way to show that one of these doesn t hold is by giving a single explicit counterexample, as in the solution above. Example Exercise 1.42 Let W be the subset of R 4 defined by: Is W a subspace of R 4? x 1 W = { x 2 x 3 R4 x 1 = x 2 }. x 4 Solution. Yes, W is a subspace of R 4. By the definition of W, 0 0 0 W 0 because 0 = 0, so 0 W. To show that W is closed under vector addition, let x, y W. By the definition of W, there exist x 1, x 2, x 3, x 4, y 1, y 2, y 3, y 4 R such that x 1 y 1 x = x 2 x 3, y = y 2 y 3, x 4 y 4 and x 1 = x 2 and y 1 = y 2. Adding these two equations gives x 1 + y 1 = x 2 + y 2. By the definition of W, this means that x 1 + y 1 x + y = x 2 + y 2 x 3 + y 3 W. x 4 + y 4 Since x, y W were arbitrary, this holds for all x, y W, so W is closed under vector addition.

42 CHAPTER 1. VECTOR SPACES To show that W is closed under scalar multiplication, let a R and x W. By the definition of W, there exist x 1, x 2, x 3, x 4, R such that x 1 x = x 2 x 3, x 4 and x 1 = x 2. Multiplying this equation by a gives ax 1 = ax 2. By the definition of W, this means that ax 1 a x = ax 2 ax 3 W. ax 4 Since a R and x W were arbitrary, this holds for all a R and x W, so W is closed under scalar multiplication. Since W contains 0, is closed under vector addition, and is closed under scalar multiplication, then W is a subspace (by Proposition 1.35 on page 36). Example Problems Example Problem 1.43 Let W be a line in R 2, meaning that there exist a 1, a 2, c R such that [ ] x1 W = { R 2 a 1 x 1 + a 2 x 2 = c}. x 2 Prove that W is a subspace of R 2 if and only if W contains the origin. Solution. To prove the forwards implication, assume that W is a subspace of R 2. Since all subspaces contain the zero vector of the full real vector space (by Proposition 1.34 on page 36), then W contains 0, which is the origin in R 2.

1.2. SUBSPACES 43 For the backwards implication, assume instead that W contains the origin. This means that x 1 = 0, x 2 = 0 satisfies the defining equation for W, so c = 0 in that equation. This tells us that W is defined as: [ ] x1 W = { R 2 a 1 x 1 + a 2 x 2 = 0}. x 2 To show that W is a subspace, we need only show that W contains 0, is closed under vector addition, and is closed under scalar multiplication (by Proposition 1.35). By assumption W contains the origin, which is 0 in R 2. [ ] [ ] x1 y1 To see that W is closed under vector addition, let, W. This means that: a 1 x 1 + a 2 x 2 = 0 and a 1 y 1 + a 2 y 2 = 0. Adding these two equations and grouping like terms, we have that: ], satisfies the defin- ] were arbitrary, this a 1 (x 1 + y 1 ) + a 2 (x 2 + y 2 ) = 0. [ ] [ ] x1 + y This tells us that 1 x1, which equals + x 2 + y 2 [ x ] 2 x1 ing equation for W and so is in W. Since and x 2 [ y1 [ y 2 y1 y 2 x 2 y 2 proves that W is closed under vector addition. [ To ] see that W is closed under scalar multiplication, let α R and x1 W. This means that: x 2 a 1 x 1 + a 2 x 2 = 0. Multiplying both sides of this equation by α and using the associativity and commutativity of multiplication of real numbers, we have that: a 1 (αx 1 ) + a 2 (αx 2 ) = 0. [ ] [ ] αx1 x1 This tells us that, which equals α, satisfies the defining equation for W and so is in W. Since α and were arbitrary, this proves αx 2 [ ] x 2 x1 that W is closed under scalar multiplication. x 2

44 CHAPTER 1. VECTOR SPACES Since W is nonempty, closed under vector addition, and closed under scalar multiplication, then W by Proposition 1.35 on page 36 it is a subspace. Putting all of this together, we have shown that a line in R 2 is a subspace if and only if it contains the origin.

1.3. SPANS 45 1.3 Spans In a real vector space V, a linear combination of vectors v 1,..., v k V is an expression of the form a 1 v 1 + + a k v k for some a 1,..., a k R. The set of all linear combinations of a finite set A of vectors is called the span of A. The span of any finite set of vectors is a subspace. Basic theory Now that we are familiar with the basics of vector spaces and their operations, we can begin to build on this understanding. The next concepts that we explore are linear combinations and spans. These concepts are so fundamental that capturing their meaning without actually using the terms themselves can be quite cumbersome. Definition 1.44 Let V be a real vector space. A linear combination of v 1, v 2,..., v k V is defined to be an expression of the form a 1 v 1 + a 2 v 2 + + a k v k, where a 1, a 2,..., a k R. In this context, the scalars a 1, a 2,..., a k are called the coefficients of the linear combination. A linear combination of the empty set of vectors is defined to be the vector 0. The linear combination 0v 1 + 0v 2 + + 0v k is called the zero linear combination of v 1, v 2,..., v k. A concept closely related to that of linear combinations is that of the span of a finite set of vectors, which we now define.

46 CHAPTER 1. VECTOR SPACES Definition 1.45 Let V be a real vector space, and let A = {v 1,..., v k } V. The span of A, denoted by Span(A), is defined to be the set of all linear combinations of the vectors in A: Span(A) = {a 1 v 1 + + a k v k a 1,..., a k R}. Since a linear combination of the empty set of vectors is by definition 0, then Span( ) = {0}. Two other ways to state that Span(A) = W are to say that A spans W or that A is a spanning set for W. Since subtraction is merely addition together with a scalar multiplication by 1 (Definition 1.20 on page 18), linear combinations can include subtraction as well as addition. Figure 1.46 is a good picture to keep in mind when working with spans. It is shown in R 2, and you should envision for yourself a similar plane being spanned by two vectors (not in the same line) in R 3. It is also helpful to imagine an analogous picture for spans in other real vector spaces, even if the picture there can t literally be drawn.