Systems of equation and matrices Jean-Luc Bouchot jean-luc.bouchot@drexel.edu February 23, 2013 Warning This is a work in progress. I can not ensure it to be mistake free at the moment. It is also lacking some information. But at least, for those of you who could not attend a class (never attended?... remember classes are mandatory normally) you get a coarse idea of what is being done. However nothing works better than doing it; practicing and asking questions when something is not clear. You will not learn playing the drums by watching random guys playing, don t expect it to be different with math. Last but not least... Don t be selfish nor shy, if you see any mistakes (shall they be in the grammar or in the math) in these notes, please let me know (either personnaly or per email) so that your fellow students can also learn from correct material. 1 Some motivation So far we were intersted with single variables being completely free, independent. However, it is sometimes (almost always?) mandatory to consider two or more variables interacting with each others. As a usecase, consider the following problem: if we know that 10 lbs of rice and 20 lbs of potatoes cost $35 and that 35 lbs of rice and 12 lbs of potatoes cost $39, can we find out the price/lb for rice and potatoes? This chapter aims at introducing some new methods to write down equations involving more than one variables as well as methods for solving these sets of equations. 2 Systems of equation 2.1 Introduction If we go back to the previous example we can denote the price/lb of rice by the free variable x and the price/lb of potatoe using the free variable y. In that case the two previous conditions read 10x + 20y = 25 and the second one reads 30x + 12y = 39 and we need to find all the points (x, y) that fulfill both conditions (or equations). Definition 1 (System of equations). If n free variables (which we shall here formally denote x 1 to x n ) are related by m equations or m conditions (like the one above), we say that we are facing a system of m equations with n variables. If all the variables appear isolated from each other (i.e. they are not multiplying each other) and without any exponent, we say that this system of equations is linear. Systems of linear equations are usually written with a curly bracket in front; in the case of the potatoes and rice problem, we get { 10x + 20y = 25 which is an example of system of linear equations. 30x + 12y = 39 1
Example 1. { 10x + 20y 2 = 25 30x + 12y = 39 and { 10x + 20y = 25 x y = 39 are examples of non linear systems... which are not studied here. Definition, mise en equation, exemples 2.2 Facts As you (might) have already seen in high school, we can picture these systems by graphing the equations. As we see the equations relate any x value to a y value by a linear equation and can hence be represented by a line. (Exactly as we did in the case of supply and demand problems) Therefore, we are looking for points which are on both lines and there are three possibilities The two lines are parallel and non overlapping there are no solutions The two lines are parallel and overlapping there are infinitely many solutions The two lines have different slopes there is only one solution: where both lines cross. y y y x x x (a) The two lines have the same slope but never cross each other (b) The two lines have the same slope and overlap (c) The two lines have different slopes Definition 2 ((in)consistent systems). A system of equations is said to be consistent if it has at least one solution. If it has no solution it is said to be inconsistent. It can be shown that, as with the case of lines, a system of n equations with n unkowns always falls into one of the following cases: 1. It is inconsistent 2. It is consistent and has infinitely many solutions 3. It is consistent and has a unique solution While the graphical method introduced above is very easy to use with two unknowns it s almost impossible to do as soon as we have more than two unknowns. In the next sections we introduce analytical methods to solve such systems with (potentially) many unknowns. 2
2.3 Substitution and echelon The first method we introduce here is something most of you did in high school. It is called the substitution or elimination method. The basic idea is relatively simple: you first solve the first equation for on of the unkown and replace in the following ones. By doing this a sufficient number of time, the system should be simplified enough to be solvable. If we consider the potatoes and rice example above and multiply the first equation by 3 and calculate the difference with the second one, we get the following equivalent system: { 48y = 36 30x + 12y = 39 As we see now, the unkown x no longer appears in the first row. It can be solved for y and we get y = 3/4 and hence replacing in the second equation, we get 30x + 12 3 4 = 39 which leads to (if we solve for x) x = 1 ($/lb of rice - try to always remember the units, it is often a good guide for solving problems) As you can see, this process gives a rather straightforward solution (which is unique!) Exercise 1. Solve the following system of equations for x, y and z. x + 2y + 3z = 5 x y + 5z = 0 2x + 2y + 2z = 2 As you can see, without a systematic way to do this, it can get hard to get to the result. Therefore an appropriate method has been developped to overcome the dimensionality of the system. This method is called the echelon method and can be described as follows: The idea is to replace a given system by a simpler and equivalent one applying appropriate transformations The approriate transformations are one of these: 1. Interchange any two rows (written as R i R j ) 2. Multiply a row by a non-zero number ( R i tr i, with t 0) 3. Add a multiple of one row to another one (R i R i + tr j ) These transformations are called appropriate because they can be reversed Use these transformations until you get to a triangular system (i.e. you get rid of an unkown at each new equation) Solve the last (easy) equation for the remaining unkown and back propagate the result in the other equations Definition 3 (Equivalent systems of equations). Two systems of equations are said to be equivalent if they have the same set of solutions Example 2. We consider the example above and solve with this systematic/algorithmic approach. We will at each new step get rid of the left most unkown (i.e. first x and then y and hence, only z should remain in the last equation - note that you should always try to keep your writting well done to clarify your results... and your mind!) Remember the system of equations reads x +2y +3z = 5 x y +5z = 0 2x +2y +2z = 2 3
We will get rid of the red part by doing R 2 R 1 + R 2 and the red one by applying R 3 R 3 2R 1 and we get the following equivalent system x +2y +3z = 5 y +8z = 5 2y 4z = 8 and we can no get rid of the last red term by R 3 R 3 + 2R 2 and so we finally get x +2y +3z = 5 y +8z = 5 +12z = 2 And now we only need to solve R 3 for z and backpropagate the result to R 2 and R 1. Hence we get x +2y +1/2 = 5 y +4/3 = 5 z = 1/6 as we see only one unkown (namely y) appears in the second equation and hence we can solve it for y and backpropagate the result in the first one. x +22/3 = +9/2 y = 11/3 z = 1/6 and we finally can solve for x which gives us the unique solution x = 17/6 y = 11/3 z = 1/6 Remark 1. As long as you follow these steps rigorously, nothing wrong can happen. However, it is sometimes better to swap to rows to have this nice triangular shape: DO NOT HESITATE! ADD EXAMPLE WITH inf solutions and no solution 3 An introduction to matrices 3.1 Definitions An interesting tool to represent such situations are matrices. They can be seen as tables of values at different locations. For the example of rice and potatoes, we would right: [ As we can see it corresponds to the coefficients of the left hand side and right hand side of the different equations concatenate together. 3.1.1 Theory Definition 4 (Matrix). A matrix is a mathematical object that gather coefficients together in order to process data more efficiently. Do not worry too much about the definition, you can think of matrices as a way to represent systems of equation in a compact way as a table of numbers. Definition 5 (Size of a matrix). A matrix that has n rows and m columns is said to be a n m matrix (number of rows first, then the number of columns) or a matrix of size n m. 4
Example 3. The matrix associated to the potatoes and rice problem is a 2 3 matrix. Definition 6. A matrix that has the same number of rows and columns is called a squared matrix. Example 4. The following matrix: is an example of a 3 3 square matrix. Definition 7 (Identity). The square matrix of size n n with 1 s on the diagonal and 0 s elsewhere is called the identity matrix of size n. Why this matrix is called identity will become clear later. Definition 8 (Entries of a matrix). An entry of a matrix is a number inside this matrix at a given location. This entry can be associated to a label (a, b) where a denotes the row number and b the column number where the cell is. We usually write A(a, b) to denote this entry in matrix A Example 5. With the matrix we have for instance A(1, 1) = 10 or A(2, 3) = 39. 3.2 Usefullness and application 3.2.1 Representation of systems [ In case of a system of equations represented as a matrix, we have to be careful that the system is well written; i.e. that the variables should appear on the left hand side of the = sign and in the same order in each equations and the constants should be on the right hand side of the = sign. Once the equation are well written, we can just erase all the = signs and unkowns and plug the resulting coefficients in a matrix/table. It is usual to separate the right hand side of the = sign from the left hand side by adding a vertical bar between the coefficients in the matrix. While this is not mandatory is helps working with matrices for solving linear systems of equations. Definition 9 (Augmented matrix). A matrix that contains the parameters of a system of equations with the vertical bar in the middle is called an augmented matrix. Hence, for the potatoes and rice problem, we would write 3.2.2 Solving a linear system: the Gauss-Jordan method The main idea of the Gauss-Jordan method is the exact same as using the echelon method. While we were trying to get a triangular shaped system of equations before, we will try, by applying the exact same row operations, to transform the matrix into a matrix containing the identity on the left side of the vertical bar. In the case of a system of 2 equations with 2 unkowns, we want to get to 1 0 a 0 1 b where a and b depend on the calculation used to transform the matrix. 5
Example 6. We still consider the potatoes and rice problem and try to solve it using the Gauss-Jordan method. So starting from the original augmented matrix we will start by getting a 1 as the red component and getting rid of the green one: R 1 1 10 R 1 and R 2 R 2 3R 1 which results in 1 48 36 0 6 31.5 where we want to get 1 instead of the red number; R 2 1 48 R 2: 1 2 2.5 0 1 0.75 Finally we only need to get rid of the last green term out of the diagonal now by applying R 1 R 1 2R 2 1 0 1 0 1 0.75 The solution is seen by simply taking the coefficients out: x = 1 and y = 0.75 (which are the same as we had before!). 3.3 Operations Here we introduce some rule to calculate and work with matrices. This rules are hard to understand and use at first but they finally become natural once we work enough with it... Practice a lot! Equality We say that two matrices are equal if their coefficients correspond everywhere. In other words, to verify the equality between two matrices we try to overlap them and make sure all the coefficients are equal. As a consequence, if two matrices are not of the same size, they cannot be equal to each other. Addition As we would do with normal numbers, we would like to define some operations between these new objects. The first one corresponds to the addition of two matrices. Addition of two matrices is done by simply adding each term of a same location together. As a consequence, if two matrices are not of the same size, they cannot be added together. Multiplication Numbers can be multiplied too. A multiplication between matrices can be defined but some care should be taken. We indeed have some strong conditions under which this is possible. In order to calculate the product A B their dimensions should fulfill the following condition (assume A has size n A m A and B has size n B m B ): m n B we sometimes say that their inner dimensions should be equal (to see this, write down the different dimensions respecting the order of multiplication: n A, m A, n B, m B... we say inner dimensions, because, m A and n B are those in the middle of the others) The result of this multiplication is another matrix of size n A m B (in other words the outer dimensions!) We have the following consequences: 1. Multiplying A by B (what we right A B) does not necessarily equal the product of B by A (B A). And in fact, they are almost never equal. 2. The fact that A B exists does not mean by any way that B A exists. In fact it is not likely at all. 3. While the order of multiplication had no importance when dealing with usual numbers, we have to be careful when dealing with matrices. 6
Care should be taken when multiplying two matrices as it is very easy to make mistakes. To calculate the entry at position (a, b) of the result of the product A B we first extract the row number a from matrix A and column b of matrix B. The idea is to calculate the product of these two objects component wise (i.e. the first element of a times the first element of b and then the second element of a with the second of b etc... until the last terms) and then sum up the results together. A visual way to calculate these products, is to write the matrices on different levels: 1 2 4 [ 1 3 2 7 0 1 5 [ 1 5 19 2 3 27 Multiplication with a number (scalar) It is also possible to multiply a matrix by a number (we usually call numbers scalars). This can be done by multiplying each entry of the matrix by this number, without any condition. This operations results in a new matrix which has the size of the original one. Remark 2 (Subtraction of matrices). We can define the subtraction A B by combining a scalar multiplication with an addition: A B = A + ( 1) B Transposition The transposition is a new operation created for matrices. It can be seen as a mirroring or flipping of the matrix. It can be applied to any matrix and creates a new matrix where the columns of the original one become the rows of the new ones. As a consequence, transposing an n m matrix results in an m n matrix. Examples Add a bunch of examples for each of them. Exercise 2 (Identity?). 3.4 Inverse of a matrix Definition 10 (Invertability and inverse). A square matrix A is said to be invertible if it exists another matrix B such that A B = I AND B I. In that case, B is called the inverse of A (and as a consequence, A is also the inverse of B). Remark 3. When dealing with a normal number a, we always have that a a 1 = 1. To mimick this notation, we write the inverse of the matrix A (if it exists) A 1. Finding an inverse Finding the inverse of a matrix A in a general case is a hard task. In case of a 2 2 matrix, we are actually looking for four numbers w, x, y, z such that w x Consider the following matrix as an example: y z 1 0 A 0 1 [ 1 2 3 7 then the previous product of matrices is equivalent to the two following systems of equations: { w + 2y = 1 3w + 7y = 0 and { x + 2z = 0 3x + 7z = 1 7
as we have seen in a previous section we can solve these systems using the Gauss-Jordan method on the augemented matrices: 1 2 1 1 2 0 and 3 7 0 3 7 1 and as we can see, the two augmented matrices have the same left hand side and hence, we will try to solve both systems at the same time and if we find an answer, this will be the inverse of the matrix A. If we cannot find an answer it means that the matrix A is not invertible. [ 1 2 1 0 3 7 0 1 note that this system can also be written as [A I. Applying the following sequence of row operations: R 2 R 2 3R 1, R 1 R 1 2R 2 yields the following equivalent augmented matrix: [ 1 0 7 2 0 1 3 1 7 2 and therefore the matrix A 1 = is the inverse we were looking for. You can check yourself that 3 1 we do indeed have A A 1 = I What we just did for the inverse of a particular 2 2 matrix can be generalised: finding the inverse matrix is equivalent to solving the following augmented matrix using the Gauss-Jacobi method: [A I [ I A 1 no sequence of row operations can be found for that, then A is not inversitble. 3.5 Application of the inverse Now that we know we can (sometimes) invert a matrix and know how to do it, we see how it can be applied to the resolution of linear systems of equations. We first notice that a system can be written as a product of the matrix by the matrix n 1 of variables: { x +2y = 3 1 2 x 3 3x +7y = 5 = 3 7 y 5 which we can also write as A X = B. Now when multiplying both sides by A 1 (which exists... see above!) to the left we get A 1 A X = A 1 B and hence, as A 1 I we get X = A 1 B 8