March 19 - Solving Linear Systems Welcome to linear algebra! Linear algebra is the study of vectors, vector spaces, and maps between vector spaces. It has applications across data analysis, computer graphics, differential equations, Fourier analysis, quantum mechanics, and of course engineering. Our course is very intensive (12 two-hour classes in just six weeks!) so we will necessarily move quickly. It is essential that you put in significant work outside class. The typical structure of each class will be half lecture and half practice in groups. You re expected to be active and prepared! Advice: Linear algebra typically marks a shift in mathematics coursework. There are more definitions than previous classes, and the material becomes more abstract. While not much more than basic arithmetic is needed to do computations, wrapping your head around ideas and figuring out what to add or multiply can at times challenging. So, definitely keep up with the terminology (knowing definitions well is the first step in studying, not the last), practice lots of exercises (more than assigned!), and talk with other people (students and during office hours). Like piano or learning a language, math is not something you pick up by watching... practice, make mistakes, and then practice more. Class goals: 1. Learn the standard tools and lingo of linear algebra 2. Apply these to real problems 3. Think and communicate like a mathematician 4. Have fun! Introduction to Linear Systems Definition A linear equation is an equation of the form a 1 x 1 + + a n x n = c, where the a i s and c are real numbers and the x i s are variables. A system of linear equations is a collection of several linear equations (using the same set of variables). Solving a system of linear equations means deciding whether the equations have a common solution. We could proceed: Algebraically: look for numbers which simultaneously satisfy all the equations. Geometrically: understand how the corresponding objects intersect. 1
Example Solve the system the high school ways (elimination/substitution) x + 2y = 1 2x + 3y = 1 Answer is x = 1 and y = 1, indicating the two lines in the plane intersect in a point. Example Consider the system x + 2y + 3z = 1 3x + 2y + z = 1 7x + 2y 3z = 1. We could try substitution/elimination, but it probably won t be organized/efficient. (It turns out this example has an infinite number of solutions... three planes are intersecting in a line). One could add yet more equations and/or more unknowns... how do we solve larger linear systems efficiently? Gauss-Jordan Procedure The Gauss-Jordan algorithm can be used to solve systems of linear equations, but we ll see it has many other uses in linear algebra too. Some basics: Definition A matrix is a rectangular array of numbers, described by its dimensions m n, denoting number of rows by columns. [ ] 2 3 We ll use capital letters for matrices. So, A = is a 2 3 matrix. 1 9 5 Definition The elementary row operations on a matrix are: Interchange two rows. Multiply/divide a row by a nonzero constant. Add/subtract a multiple of one row from another. Given a matrix, row operations produce other matrices according to three rules. The motivation is these correspond to three operations that change a system of linear equations into one with the same solution set: interchange two equations; multiply one equation by a nonzero constant; add a multiple of one equation to another. Definition The Gauss-Jordan process (a.k.a. row reduction algorithm a.k.a. Gaussian elimination) for a matrix is as follows: Start with the leftmost nonzero column. By using elementary row operations, 2
1. Get a nonzero entry in the column s top row (this is called a pivot ). 2. Get zeros in all entries directly below the pivot. 3. Move down from the pivot to the next row and repeat the previous two steps while ignoring/covering all rows above this new row (to be clear, your first step will now be to move over to the right one column, since you are now in a column of all zeros after covering up the higher rows) 4. Repeat until there are no more rows to address. The result of this process is called a row echelon form of the matrix. [ ] 3 3 Example Put A = in row echelon form. 1 2 4 [ ] [ ] [ ] 3 3 I II 1 2 4 II 3I 1 2 4 1 2 4 3 3 3 12 (While the I II interchange wasn t necessarily, it avoided the use of fractions! You could also avoid fractions by 1 I as your first step.) 3 To be clear, the row echelon form satisfies 1. All nonzero rows are above all rows entirely of zeros. 2. If a row has a pivot, then each row above it has a pivot farther left. Definition If in addition, each pivot position in a row echelon form has been made to be a 1 and all entries in each pivot column other than the pivot are zeros (not just those below the pivot), the result is called the reduced row echelon form. Example Continuing the above example, find that [ ] [ ] 1 2 4 1 3 II 1 2 4 3 12 1 4 I 2II [ 1 ] 4 1 4 While there are many possible row echelon forms for a matrix, the reduced row echelon form of a matrix is in fact unique. [ ] 3 Example Put in row echelon form. Also, find the reduced row echelon form. 2 4 Answer: You could interchange [ ] rows I and II to get a row echelon form. The 1 reduced row echelon form is. 1 Example Solve the system x + 2y = 1 2x + 3y = 1 by making an augmented matrix and find the row echelon form and reduced row echelon form. How easy is it to read off the solution at various stages? 3
Gauss-Jordan Practice Augmented matrices To a linear system with m equations and n variables we ll associate an augmented matrix of dimensions m (n + 1). The first n columns are all the coefficients of the n variables and the last (augmented) column consists of the constants. Example The linear system gives rise to the augmented matrix x 1 3x 2 + 4x 3 = 4 3x 1 7x 2 + 7x 3 = 8 4x 1 + 6x 2 x 3 = 7 1 3 4 4 3 7 7 8 4 6 1 7 Now find the row echelon form and read off the solution to the system. Answer: The reduced row echelon form turns out to be 1 3.5 1 2.5 1 Since the last row reads = 1, there is no solution to the system. This examples illustrates a general fact: A linear system has no solution if and only if the reduced row echelon form of the corresponding augmented matrix has a row that corresponds to the equation = 1 (no matter what values we give the variables, we can never make = 1 true). Definition A linear system that has no solution is called inconsistent. A linear system that has a solution is called consistent (it will have either a unique solution or infinitely many solutions). Finding the row echelon form of the augmented matrix for a system is enough to decide if the system is consistent (it is inconsistent only if there is a row corresponding to the equation = c where c ). However, when the system is consistent it is convenient to go all the way to the reduced row echelon form in order to read off the simplest form of the answer. 4
Example Solve the system x 1 +2x 2 +2x 4 +3x 5 = x 3 +3x 4 +2x 5 = x 3 +4x 4 x 5 = x 5 = Answer: Find x 3 = x 4 = x 5 = and x 1 = 2x 2, where x 2 may be any value. Definition A basic variable is a variable whose value is fully determined in the solution, while a free variable is a variable whose value may be chosen freely. Notice that basic variables correspond to pivot columns, while free variables correspond to non-pivot columns. Example Given the augmented matrix 1 4 2 3 1 3 1 1 confirm that it is in rref and write down the solution in parametric form by introducing a parameter for each free varirable. What does this describe geometrically? Answer: x 5 = 1 x 4 = free x 3 = 3x 4 x 2 = free x 1 = 3 + 2x 4 4x 2 Looking toward next lecture, if we let x 4 = t and x 2 = s, a very nice way to write the general solution in the vector form x 1 3 2 4 x 2 x 3 x 4 = + t 3 1 + s 1 x 5 1 We have a parametrized equation for the plane of solutions to the system. 5