ftdt. We often drop the parentheses and write Tf instead of Tf. We could also write in this example Tfx 0

Transcription

1 Linear algebra: The study of linear transformations on vector spaces. Q: What s a linear transformation? What s a vector space? Linear transformations and matrices: We first discuss linear transformations and how they lead to matrices when transforming n dimensional vectors to m dimensional vectors. Here we develop matrices through the idea of a linear function, or transformation, or mapping. These words are used synonymously. We all know what a function is: there is a domain, the function acts on an element of the domain and the result is an element of the range. Sometimes we write y fx, giving names to x in the domain, y in the range, and the function f that "turns" x into y. We sometimes express this sybolically as x f y. In this sense the element of the domain is "transformed" to an element of the range. You can see the usefulness of the word "transformation" when the domain consists of functions. So, for instance, if our domain D consists of continuous functions, we can consider the transformation T which acts on a function f via integration so that the result Tf is another function g Tf defined by gx x ftdt. We often drop the parentheses and write Tf instead of Tf. We could also write in this example Tfx x ftdt as our definition of the transformation T. Using the word "transformation" allows us to avoid the awkwardness of talking about a function of functions and instead talk about a transformation of functions as in this example. What makes a transformation linear? First, the domain has to be a vector space, things you can add and multiply by scalars. Otherwise, linearity doesn t make sense. Note that you can 3 add traditional vectors such as and multiply by scalars, e.g You can also add functions and multiplty them by scalars, such as px 3x 4, where we add the functions x and after multiplying by the scalars 3 and 4 respectively. On the other hand, if your domain is words (or, more precisely, finite strings of characters), then there is not an obvious way to add them and multiply by scalars. There are plenty of transformations you can define, however, on such a domain, e.g. if s is a string you can have Ts #s, the number of letters in the string, or Ts ss (the string s concatenated with itself), etc. But there is no such thing as a linear function on this domain (at least not without some more work...) Once our domain is a vector space we define a linear transformation this way: Definition: T is linear if for everyū, v in the domain, and every scalar c, we have Tū v Tū Tv and Tcū ctū. You can do this in one shot: T is linear if for everyū, v in the domain, and every choice of scalars a, b we have Taū bv atū btv. (the operation of T "distributes through" the linear combination) Linear transformations from R n to R m

2 Here we study transformations from the domain of n dimensional real vectors R n that result in an m dimensional vector in R m. A vector, let s call it x, in, let s say, R 3, is a vector with three real components, that you can write x,, x 3 or,, x 3 or. The idea of a vector in R 3 is that it is "an ordered three-tuple" of real numbers with addition and scalar multiplication defined in the usual way. How we write it is a matter of convention, but it will x 3 be convenient to denote vectors in R n using "column vectors" i.e. x. We assume the usual operations of vector addition and scalar multiplication as given. Now, what would a transformation froom R 3 to R 2 look like? Well here is one: We define the transformation T, with y Tx, by y 2 x 3 or y 2 x 3 y 2 x 3 y Tx. This transformation T is, however, not linear. How y 2 x 3 do you know?? Well, you could argue T T T so the required property of linearity is not satisfied. This next transformtion though, is linear: Tx T 4 3 x 3 x 3 While it s "pretty clear" that it s linear we ll have to carefully show that later. For now we just say: If T is a linear transformation from R n to R m that is linear, and we denote y Tx then the transformation must have the form: y a a 2...a n y 2 a 2 a 22...a 2n y m a m a m2...a mn where the coefficients a ij, i, 2,.., m, j, 2,.., n are fixed scalars. By convention we associate a ij as the coefficient of x j in the formula for y i. 2

3 As an example from R 3 to R 2 we might have y 3 2 5x 3 y 2 4 2x 3 Note that such a linear transformation is completely defined by the coefficients a ij. These coefficients form a "matrix", a rectangular array of scalars for which we will define certain operations. By convention, the system of equations y a a 2...a n y 2 a 2 a 22...a 2n y m a m a m2...a mn is written in vector/matrix form (or just matrix form) as y a a 2 a n y Ax where y y 2 a 2 a 22 a 2n Ax y m a m a m2 a mn This defines the operation of a matrix times a column vector. By convention we say that a matrix with m rows and n columns is mn (pronounced "m-by-n"). Given a matrix A, the element sitting in row i and column j is denoted by a ij, or A ij which is sometimes more convenient. This is consistent with the notation above. Mathematical formula for a linear transformation: If we look at y a a 2...a n y 2 a 2 a 22...a 2n y m a m a m2...a mn we can see that for a generic row i, we would have y i a i a i2...a in. This in turn can be written as a sum: n y i a ij x j, where the sum is over the index j and the index i is fixed. Mathematically, j sums are a more precise (and compact) way of defining expressions like this, as opposed to y i a i a i2...a in and you will often see such sums in the text. 3

4 Matrix times a vector: describing the operation ) The expression y i a i a i2...a in can be thought of as a dot product of two a i a i2 vectors y i row i x Here, we re not too concerned about a in the shape (row/column) of the vectors, because we re thinking of them as vectors. 2) The fundamental operation in matrix multiplication is a row times a column, namely a i a 2i a in a i a i2...a in y i so we usually think: If y Ax then y i is the (matrix) product of row i with x. 3) Ax as a linear combination of the columns : It s pretty easy to see (just check that everything is there) that a a 2...a n a a 2 a n y a 2 a 22...a 2n a 2 a a 2n a m a m2...a mn a m a m2 a mn col... Thus we say: Ax is a linear combination of the columns, with the components of x as coefficients. This last form is particularly useful and important. Here are several consequences: It is convenient for us to introduce in R n what we call the standard basis vectors: 4

5 ē,ē 2,...,ē n so thatē j has a "" in its j th component and is otherwise zero. In R 3 we use the standard namesē ī, ē 2 j andē 3 k for these vectors, and refer to them as "coordinate vectors". A mathematical definition of these standard basis vectors is ē i j for i j, if i j ) Confirmation of linearity: Aū v Aū Av, Acū caū We have Aū Av u col u 2...u n v col v 2...v n u v col u 2 v 2...u n v n Aū v The second result can be obtained similarly Acū cu col cu 2...cu n c u col u 2...u n caū 2) Aē col, Aē 2,..., Aē n and in general: Aē j col j 5

6 This is easy to see but one could just argue that Aē j is a linear combination of the columns, with all coefficients except for the coefficient of col j and the coefficient in that case is, so that Aē j... col j.... Looked at from another viewpoint, this result shows us that if you had an unknown transformation Tx Ax then the columns can be obtained, respectively, from the values Tē, Tē 2,.., Tē n 3) Every linear transformation from R n to R n is of the form Tx Ax for some matrix A. Suppose T is linear. Note first that in general x ē ē 2... ē n so that Tx T ē ē 2... ē n and by the assumption of linearity, Tx T ē ē 2... ē n Tē Tē 2... Tē n Tē Tē 2 Tē n Ax with A Tē Tē 2 Tē n. In words, the columns are the values of T on the standard basis vectors of R n. Systems of linear equations: A system of linear equations, with m equations in n unknowns, takes the form a a 2...a n b a 2 a 22...a 2n b 2 a m a m2...a mn b m This can be written in the more compact form Ax b. Our goal a bit later is to characterize solutions of a system in terms of properties of the matrix A. Matrix operations: We define the basic matrix operations - matrix addition, matrix multiplication, scalar multiplication - by thinking of a matrix as (defining) a transformation, or function. Matrix addition is defined through the sum of two functions; matrix multiplication is defined by thinking of the composition of two linear transformations; and likewise for scalar multiplication. Matrix addition: If we consider the tranformation Tx Ax Bx (where A and B are the same 6

7 size) then it is easy to see that T is a linear transformation, and hence can be written as Tx Cx for some matrix C. We define A B to be that matrix C, and it is easy to see that C ij A B ij A ij B ij and this defines A B from the entries in A and B. Note that from the very definition B we have Ax Bx A Bx Matrix multiplication: Suppose we have two transformations. If we consider the composition of the transformations Tx ABx (assuming the sizes are correct so that A acts on the result of Bx ) it is easy to see that this is a linear transformation either by imagining what happens to the components of x or by observing Tū v ABū v ABū Bv ABū ABv Tū Tv and similarly Tcū ABcū AcBū cabū ctū. Thus there is a single matrix C such that Tx Cx and we define the matrix product AB to be that matrix C, so that we have Tx Cx ABx ABx from the definition B. Now, what is that matrix C? We have ABx A col of B of B... of B A col of B A of B... A of B A col of B A of B A of B ABx So we observe: The j th column B is A col j of B. Now, what is AB ij. This element is the i th component of the j th column B and hence the i th component col j of B which in turn is row i col j of B. We should set this off: AB ij row i col j of B Let s give names to the sizes. If A is mn then B needs n rows so let s say B is then n p. Mathematically, thinking of the row*col above as a dot product, we can write the sum n n AB ij a ik b kj where the sum is over the index k. The elements a ik k are the elements k n of row i and the elements b kj k are the elements of column j of B and the sum multiplies corresponding entries and adds. Three ways to view matrix multiplication: Above we see the following a) The columns B are A times each columns of B, in turn. Each column B is a combination of columns. b) The i, j element B is row i times column j of B. c) The rows B are each row times B, in turn. Each row B is a linear combination of the rows of B. The viewpoint in c) is new - it is easy to see that each row B involves taking the corresponding row and multiplying by each column of B in turn. On the other hand, just as in the case of a matrix times a vector, here, a row times a matrix can also be written as a 7

8 linear combination of the rows of the matrix, with the coefficients the elements of the row. Here are the two viewpoints: or Scalar multiplication: Here we want to define ca. We consider the transformation Tx Acx Cx for a single matrix C that we define as ca. It is easy to see that C ij ca ij ca ij is how we must define C in order for this to work. We have defined all the matrix operations. We should note that a column vector x acts just like a column matrix whenever it interacts with other vectors or matrices and so a vector x can be considered as a matrix x whenever we want. So, for instance Ax is the same as Ax if we consider x as a matrix and use the definition of matrix multiplication to perform the second operation. All of the properties of matrix multiplication follow easily by considering the related transformations: ) ABC ABC the associative law Consider the transformation ABCx 8

9 ABCx ABCx from the definition of BC. ABCx ABCx from the definition B The fact that ABCx ABCx for all x means that BC ABC must hold 2) AB C AB AC the distributive law Consider the transformation AB Cx. We have AB Cx ABx Cx from the definition of B C ABx Cx ABx ACx from linearity of multiplication by A ABx ACx ABx ACx from the definition of matrix multiplication ABx ACx AB ACx from the definition of matrix addition Since AB Cx AB ACx for all x, the matrices multiplying x must be the same. The other laws for how matrix operations interact can be shown similarly. 9