MTH 232 Semester II 2-2 Linear Algebra Reference Notes Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2
ii
Contents Table of Contents iii Systems of Linear Equations. Solve Systems of Linear Equations.............................2 Gaussian Elimination.................................... 7.3 Theory of Solutions......................................4 Homogeneous Systems................................... 7 2 Matrix Algebra 2 2. Introduction and Examples................................ 2 2.2 Matrix Operations..................................... 23 2.3 The Inverse of a Matrix.................................. 27 2.4 Solve Matrix Equations by Inverse............................ 3 2.5 Rank............................................. 35 2.6 Find Determinants..................................... 37 3 Eigenvalues and Eigenvectors 39 3. Eigenvalues and Eigenvectors............................... 39 3.2 Characteristic Equation.................................. 4 3.3 Diagonalization....................................... 44 3.4 Application of Diagonalization.............................. 48 4 Vector Spaces 49 4. Vectors........................................... 49 4.2 General Solution in Vector Form............................. 52 4.3 Vector Spaces and Subspaces............................... 56 4.4 Subspaces of a Matrix................................... 6 4.5 Linear Independence.................................... 65 4.6 Basis of R n......................................... 69 5 Orthogonality 73 5. Inner Product Spaces.................................... 73 5.2 Orthogonality........................................ 75 5.3 Orthogonal Projection................................... 8 5.4 Gram Schmidt Orthogonalization............................. 83 5.5 Least-Squares Problems.................................. 85 iii
Chapter Systems of Linear Equations. Solve Systems of Linear Equations In this chapter we first review how systems of linear equations involving two variables are solved using the method of substitution learn in secondary school elementary algebra. (Basic properties of linear equations and lines should be reviewed.) Because the method of substitution is not suitable for linear systems involving large numbers of equations and variables, we then turn to a different method of solution involving the concept of an augmented matrix, which arises quite naturally when dealing with larger linear systems. We then study matrices and matrix basic operations in their own right as a new mathematical tool. With these new operations added to our mathematical toolbox, we return to systems of linear equations from a fresh point of view. A new technique, called the elementary row operations, will play an important role. Solution of systems of linear equations has its own importance in applications and they appear frequently in many practical problems. We briefly review several real life problems in this section. To establish basic concepts, we consider the first simple example. Example.. (Systems with two variables) If 2 adult tickets and child ticket cost $8, and if adult ticket and 3 child tickets cost $9, what is the price of each? Solution Let x be the price of adult ticket and y the price of child ticket. Then 2x + y = 8 and x + 3y = 9. We now have a system of two linear equations with two variables. It is easy to find ordered pairs (x, y) that satisfy one or the other of these equations. For example, the ordered pair (4, ) satisfies the first equation, but not the second, and the ordered pair (6, ) satisfies the second, but not the first. To solve this system, we must find all ordered pairs of real numbers that satisfy both equations at the same time. The set of all such ordered pairs is called the solution set. The method of substitution works nicely for systems involving two variables. By eliminating one variable from these two equations will immediately solve the problem. For example, if we choose to eliminate y, then we may substitute y = 8 2x (follows from the first equation) into the second equation. That gives x + 3(8 2x) = 9. After solving, we have the solution x = 3 and hence y = 2. In the following we give another example: The quantity of a product that people are willing to buy during some period of time depends on its price. Generally, the higher the price, the less the demand; the lower the price, the greater the demand. Similarly, the quantity of a product that a supplier is willing to sell
. Systems of Linear Equations during some period of time also depends on the price. Generally, a supplier will be willing to supply more of a product at higher prices and less of a product at lower prices. The simplest supply and demand model is a linear model where the graphs of a demand equation and a supply equation are straight lines. Example..2 (Supply and demand) Suppose that we are interested in analyzing the sale of cherries each day in Hong Kong. Using special analytical techniques (regression analysis) and data collected, an analysis arrives at the following pricedemand and price-supply models: p = 2q + 4, (Price-demand equation, consumer) p =.7q + 7.6, (Price-supply equation, supplier) where q represents the quantity of cherries in thousands of pounds and p represents the price in dollars. For example, we see that consumers will purchase thousand pounds (q = ) when the price is p = 2() + 4 = 2 dollars per pound. On the other hand, suppliers will be willing to supply 7.74 thousand pounds of cherries at 2 dollars per pound (solve 2 =.7q + 7.6). Thus, at $2 per pound the suppliers are willing to supply more cherries than consumers are willing to purchase. The supply exceeds the demand at that price and the price will come down. At what price will cherries stabilize for the day? That is, at what price will supply equal demand? This price, if it exists, is called the equilibrium price, and the quantity sold at that price is called the equilibrium quantity. The result could also be interpreted geometrically. The point where the two straight lines (two curves in general) for the price-demand equation and the price-supply equation intersect is called the equilibrium point. How do we find these quantities? Solution We solve the linear system p = 2q + 4, (Demand equation) p =.7q + 7.6 (Supply equation) by using the method of substitution (substituting p = 2q + 4 into the second equation) 2q + 4 =.7q + 7.6, 2.7q = 32.4, q = 2 thousand pounds. (equilibrium quantity) Now substitute q = 2 back into either of the original equations in the system and solve for p (we choose the first equation) p = 2(2) + 4 = 6. dollars per pound. (equilibrium price) If the price is above the equilibrium price of $6. per pound, the supply will exceed the demand and the price will come down. If the price is below the equilibrium price of $6. per pound, the demand will exceed the supply and the price will rise. Thus, the price will reach equilibrium at $6.. At this price, suppliers will supply 2 thousand pounds of cherries and consumers will purchase 2 thousand pounds. The substitution method used in the previous examples is an algebraic method that is easy to use and provides exact solutions to a system of two equations with two variables, provided that solutions exist. We now define some terms that we can use to describe the different types of solutions to systems of equations that we will encounter. Solutions may exist or may not exist. See the following definition. Definition A system of linear equations is called consistent if it has one or more solutions and inconsistent if no solutions exist. Furthermore, a consistent system is said to be independent if it has exactly one solution (often referred to the unique solution) and dependent if it has more than one solution (referred to infinitely many solutions). Two systems of linear equations are equivalent if they have the same solution set. 2
. Solve Systems of Linear Equations Referring to the system in Example.., the system is a consistent and independent system with exactly one solution x = 3, y = 2. A natural question may arise: Can a consistent system have exactly two solutions? Exactly three solutions? The answer is simply NO. For example, by geometrically interpreting a system of two linear equations with two variables, we gain useful information about what to expect in the way of solutions to the system. In general, any two lines in a coordinate plane must intersect in exactly one point, be parallel, or coincide (have identical graphs). In fact there are only three possible types of solutions for systems of two linear equations in two variables. These ideas are illustrated geometrically in the following. Given a system of linear equations, we will have one of the three possibilities for the number of solutions.. Exactly one solution (i.e., a unique solution). 2. No solution. 3. Infinitely many solutions (i.e., non-unique solutions). Let us first review what are solutions to systems of two linear equations with two variables and its geometrical interpretation. Geometrical interpretation of solutions The pair x =, x 2 = 3 is a solution of the following system of linear equations { 2x x 2 =, x + x 2 = 2 because if we replace the variables by the given values, then the two equalities hold. Another pair x = 2, x 2 = 2 is not a solution because the first equality does not hold. x 2 (, 3) x + x 2 = 2 (2, 2) x 2x x 2 = Figure.: Unique solution The two linear equations represent two straight lines on the (two-dimensional) plane. The solution (x, x 2) = (, 3) of the system of the two equations is the intersection of the two lines. In particular, we can find exactly one solution. The following system of linear equations { 2x x 2 =, 4x 2x 2 = 2 represents two different parallel lines. Since the two lines never meet, the system does not have any solution. 3
. Systems of Linear Equations x 2 4x 2x 2 = 2 x 2x x 2 = Figure.2: No solution The following system of linear equations { 2x x 2 =, 4x 2x 2 = 2 represents two identical lines. Any point on the line is a solution to the system. In particular, the system has infinitely many solutions. x 2 2x x 2 = 4x 2x 2 = 2 x Figure.3: Infinite number of solutions The geometrical interpretation of the solution of linear equations can be extended to more variables and more equations. For example, one linear equation with three variables a x + a 2x 2 + a 3x 3 = b is represented by a plane in the (three-dimensional) space. A system of two linear equations with three variables is represented by two planes. The following are the possible intersections (= solution to the system) of the two planes. line of solutions no solution plane of solutions 4 Figure.4: Intersection of two planes
. Solve Systems of Linear Equations In Figure.4, we observed that a system of 2 linear equations with 3 variables cannot have a unique solution. In that case, if the system is consistent, it must have infinitely many solutions. In the following we also give some examples of linear systems of more than two variables. Just remark that no professional knowledge in chemistry nor physics is needed for this elementary course MATH 3, the following two examples are only used for illustrating the wide applicability of linear systems to real life examples. Example..3 (Electric network) In the following electric network, 5Ω P 2Ω 2V + 2Ω V + i i 2 i Ω 3 Q Figure.5: An electric network the currents i, i 2, i 3 satisfy the equations Node P : i +i 2 i 3 =, Node Q : i i 2 +i 3 =, Left loop : 5i +2i 2 = 2, Right loop : 2i 2 +3i 3 =. Example..4 (Linear equations in four variables) In order to balance the following chemical reaction x CH 4 + x 2 O 2 y CO 2 + y 2 H 2O, the number of molecules x, x 2, y, y 2 must satisfy x y =, 4x 2y 2 =, 2x 2 2y y 2 =. Remind you that there is no geometrical meaning to the system of more than three variables. Referring to the systems in Examples..3..4, the first system (four equations with three variables) is a consistent and independent system with the unique solution: i = 78/3, i 2 = 5/3, i 3 = 37/3. The second system (three equations with four variables) is a consistent and dependent system with an infinite number of solutions. Remark that for these two examples the method of substitution seems to be not as efficient as for the system of two equations with two variables. Students may try it out. Indeed we compute the solutions by a more general and efficient method called the Gaussian elimination. This method will be discussed in Section.2 (page 7). But before we go into the details of this new method we need to borrow some terminologies in matrices and express the problem in another point of view. Let us recall that a linear equation is a x + a 2x 2 + + a nx n = b. 5
. Systems of Linear Equations We call a, a 2,, a n the coefficients of the equation, and x, x 2,, x n the variables of the equation. A system of linear equations is a list of linear equations. For example, a system of four equations with three variables (we say a four by three system ) is a x + a 2x 2 + a 3x 3 = b, a 2x + a 22x 2 + a 23x 3 = b 2, a 3x + a 32x 2 + a 33x 3 = b 3, a 4x + a 42x 2 + a 43x 3 = b 4. The system is completely characterized (besides the notations for the variables) by the following rectangular array of numbers called the coefficient matrix a a 2 a 3 A = a 2 a 22 a 23 a 3 a 32 a 33 a 4 a 42 a 43 and the right side b b = b 2 b 3. b 4 The system also corresponds to the augmented matrix We emphasize the following correspondence a a 2 a 3 b [ ] A b = a 2 a 22 a 23 b 2 a 3 a 32 a 33 b 3. a 4 a 42 a 43 b 4 (.) columns of A variables, rows of [ A b ] equations. (.2) Example..5 (Augmented matrix) The augmented matrices of the systems in Examples..3 and..4 are respectively 5 2 2 2 3 and 4 2. 2 2 In terms of matrices, the system (.) is equivalent to simply Ax = b, where x = (x, x 2, x 3). Next we review some basic operations of matrices which will be useful to our later discussion of solving general system Ax = b by Gaussian elimination. In Chapter 2 we will confine our attention to the world of matrices. Now, we will focus on the techniques of solving systems of linear equations. 6
.2 Gaussian Elimination.2 Gaussian Elimination The method of substitution may work well for systems involving two variables. However, it is not easily extended to larger systems. In fact, the most efficient and practical way for solving systems of linear equations is through an algorithm called Gaussian elimination. The idea is to simplify the system by creating as many zeros as in the coefficients. The solution may then be easily deduced from the simplified system. The Gaussian elimination is probably the most important method of solution. It readily generalizes to larger systems and forms the basis for computer-based solution methods. As a matter of fact, all the important computer packages nowadays for solving systems of linear equations are based on Gaussian elimination. If you have learned before to use the Cramer s rule to solve systems of linear equations, then you should forget this old method as soon as possible. Cramer s rule is a limited method because of the following:. The computation of determinants is much more complicated than elimination. 2. Cramer s rule cannot solve underdetermined (such as 2 equations, 3 variables) or overdetermined (such as 3 equations, 2 variables) systems. Gaussian elimination has no such limitations and indeed it works for any linear systems of m equations with n variables, where m < n, m = n, m > n are all permissible. Now, with some knowledge of matrix notations and operations (details can be found in Chapter 2) we are going to introduce the method of Gaussian elimination formally. But, first of all, we would also like to give you the insight of how and why the method works. Let s read the very simple example. Example.2. (Equivalent equations) To solve an equation such as 2x 5 = 3, we perform permissible operations on the equation until we reach an equivalent equation whose solution is obvious. 2x 5 = 3, 2x 5 + 5 = 3 + 5, 2x = 8, 2x 2 = 8 2, x = 4. Recall that we added 5 to both sides in the second step and divided both sides by 2 in the fourth step. These two operations simply convert the equation 2x 5 = 3 to an equivalent equation x = 4. The solution follows as we wish. The following theorem indicates that we can solve systems of linear equations in a similar manner. Theorem.2. (Three operations that produce equivalent systems) A system of linear equations is transformed into an equivalent system if (A) Two equations are interchanged. (B) An equation is multiplied by a nonzero constant. (C) A constant multiple of one equation is added to another equation. Any one of the three operations in Theorem.2. can be used to produce an equivalent system. The basic principle here is to transform a given (complicated) system into an equivalent but simpler system through a series of operations. Based on this principle we have to eliminate as many variables as possible. 7
. Systems of Linear Equations A system Ax = b of linear equations is completely characterized by its augmented matrix [ A b ], as we mentioned in Section. (page 6). Gaussian elimination can then be considered as a manipulation on the augmented matrix. In the following examples, we indicate the relation by writing systems and augmented matrices side by side. Theorem.2. is used in the examples that rows of [ A b ] are equivalent to equations. Example.2.2 (2 2 system with unique solution) Solve the system of linear equations { 2x x 2 =, x +x 2 = 2. Solution Let us solve the system { 2x x 2 =, x +x 2 = 2. [ 2 ] 2 We do the row operation 2R 2 + R (add twice of second equation/row to the first equation/row) to obtain { [ ] x 2 = 3, 3 x +x 2 = 2. 2 We may further do R R 2 (exchange the first and the second equations/rows) so that the equations are listed from the most complicated to the simplest { [ ] x +x 2 = 2, 2 x 2 = 3. 3 Substituting x 2 = 3 into the first equation, we get x = x 2 2 =. Example.2.3 (3 3 system with unique solution) Solve the system of linear equations x +x 2 +x 3 =, 2x +3x 2 +2x 3 =, 3x +8x 2 +2x 3 = 2. Solution Let us solve the system x +x 2 +x 3 =, 2x +3x 2 +2x 3 =, 2 3 2 3x +8x 2 +2x 3 = 2. 3 8 2 2 We do 2R + R 2 and 3R + R 3 to get x +x 2 +x 3 =, x 2 =, 5x 2 x 3 =. Then we do 5R 2 + R 3 to get x +x 2 +x 3 =, x 2 =, x 3 = 4. 5 4 This corresponds to x 2 = and x 3 = 4. Substituting them into the first equation, we have x = 6. 8
.2 Gaussian Elimination Example.2.4 (3 3 system without solution) Solve the system of linear equations x +x 2 +x 3 = 2, x +3x 2 x 3 = 4, x 2 x 3 = 2. Solution We simplify the augmented matrix as follows 2 2 3 4 R +R 2 2 2 2 2 2 2 2 R 2 R 2+R 3 2 2. Interpreted as a system of linear equations, we have x +x 2 +x 3 = 2, x 2 x 3 =, =. The last equation is a contradiction which implies that the system has no solution. Example.2.5 (3 3 system with non-unique solutions) Solve the system of linear equations x x 2 +3x 3 = 3, 3x +x 2 +x 3 = 5, x 2 2x 3 =. Solution We simplify the augmented matrix as follows 3 3 3 3 3 5 3R +R 2 4 8 4 2 2 3 3 4R 3 +R 2 2 R 2 R 3 3 3 2. Interpreted as a system of linear equations, we have x x 2 +3x 3 = 3, x 2 2x 3 =, =. The last equation is redundant. From the second equation, we have x 2 = 2x 3. Substituting this into the first equation, we have x = (2x 3 ) 3x 3 + 3 = x 3 + 2. The system has non-unique solutions (x, x 2, x 3) = ( r + 2, 2r, r), for any real number r. 9
. Systems of Linear Equations To summarize, the Gaussian elimination is done through the following three operations. Table.: Three (row) operations used in the Gaussian elimination. notation rr i + R j rr i R i R j description adding r times the i-th equation/row to the j-th equation/row multiplying r to the i-th equation/row exchanging the i-th equation/row with the j-th equation/row When applied to matrices, the three operations are also called row operations. The three operations do not change the solutions of the system of equations. By making use of the three operations, the system may eventually be simplified to the following upper triangular shape.... Then one may solve the system by backward substitution (i.e., solving the system from bottom up, and finding the values of more and more variables). Example.2.6 (4 3 system with unique solution) Solve the system of linear equations x x 2 +x 3 =, x +x 2 x 3 =, x +4x 2 = 5, 2x 2 +3x 3 =. Solution Let us solve the system. We simplify the augmented matrix as follows 4 5 4 5 2 3 2 3 3 62 2 3 2 3 2. The last row is redundant while the third row implies that x 3 = 2. Substituting this value into the second row, we have x 2 = 5/2. Substituting the values of x 2 and x 3 into the first row, we have x = /2. Hence the system has the unique solution (x, x 2, x 3) = ( 2, 5 2, 2).
.3 Theory of Solutions.3 Theory of Solutions Having discussed how to simplify a system of linear equations, we may ask: how simple can a system be reduced to? The answer to the question can tell us a lot of information about the solutions of the system, particularly the existence and uniqueness of solutions. Row Echelon Form In Examples.2.2,.2.3,.2.4, and.2.5, the systems (or rather, the augmented matrices [ A b ] ) are reduced to the following shapes [ ],,,, (.3) where are nonzero numbers, and are any numbers. These are the simplest shapes you can get by the three row operations. These are called the row echelon forms of [ A b ]. Generally speaking, a matrix is called in row echelon form if the following two conditions hold (where a leading nonzero entry of a row of the matrix is the first nonzero element in the row counted from the left): () All zero rows, if any, are at the bottom of the matrix. (2) Each leading nonzero entry in a row is to the right of the leading nonzero entry in the preceding row. Example.3. (Row echelon form) The following are all the possible row echelon forms of 2 2 matrices ( simplest shapes of 2 linear equations with variable). [ ] [ ] [ ] [ ],,,. The following are all the possible row echelon forms of 3 3 matrices ( simplest shapes of 3 linear equations with 2 variables).,,,,,,. In a row echelon form, the entry denoted by is located further and further to the right when one goes from the top to the bottom. We call these entries pivots of the matrix, and the columns containing the pivots the pivot columns of the matrix. Example.3.2 (Row echelon form) The following is the row echelon form for the augmented matrix in Example.2.4. The pivots are the (, )-, (2, 2)-, and (3, 4)-entries. The pivot columns are the first, second, and the fourth columns.
. Systems of Linear Equations Example.3.3 (Row echelon form) The following is a more complicated row echelon form. The pivots are the (, 2)-, (2, 4)-, (3, 5)-, and (4, 7)-entries. The pivot columns are the second, fourth, fifth, and the seventh columns. Example.3.4 (Non-row echelon form) The matrix 2 is not in row echelon form because the for the second row is after the for the third row. If we exchange the second and the third rows (so that the matrix is further simplified), then we do get a row echelon form. Similarly, the matrix 2 is not in row echelon form because the for the second row is not before the for the third row. Exchanging rows will never give us a row echelon form. We have to use row operations such as 2R 3 + R 2 and R 2 R 3 (or simply one operation R2 + R3) to further simplify the matrix in order to get a row echelon form. 2 Reduced Row Echelon Form Row echelon form is the simplest shape one can get by performing row operations. What is the ultimate simplest matrix one can get? Example.3.5 (Reduced row echelon form) The system in Example.2.3 (page 8) has been simplified to the following row echelon form. 4 We reduce the coefficient of (3, 3)-pivot to by doing R 3. 4 2
.3 Theory of Solutions We cancel entries above the (, 3)-pivot by doing R 3 + R 5. 4 We cancel entries above the (, 2)-pivot by doing R 2 + R 6. 4 Interpreted the above (reduced) row echelon form as a system of linear equations, we have x = 6, x 2 =, x 3 = 4, which immediately gives the unique solution of the original system. Example.3.6 (Reduced row echelon form) The system in Example.2.5 has been simplified to the following row echelon form 3 3 2. We use the (2, 2)-pivot to cancel entries above it by doing R 2 + R 2 2. Interpreted the above (reduced) row echelon form as a system of linear equations, we have x +x 3 = 2, x 2 2x 3 =, =, which gives the non-unique solutions of the system x = r + 2, x 2 = 2r, x 3 = r, with r arbitrary. In similar ways, we may further reduce the row echelon forms (.3) to the following [ ],,,, where are reduced to, the entries above are reduced to, and are again any numbers. These are the simplest matrices you can get by the three row operations, and we call them the reduced row echelon forms of [ A b ]. 3
. Systems of Linear Equations Generally speaking, a matrix is called in reduced row echelon form if it is in row echelon form, that is, if it satisfies the two conditions () and (2) (on page ), and if it satisfies the following additional two conditions: (3) Each pivot (leading nonzero entry) is equal to. (4) Each pivot is the only nonzero entry in its column. By comparing Examples.3.5,.3.6 with Examples.2.3,.2.5, we observe that the advantage of reduced row echelon form is that we can write down the solutions directly, without any backward substitution. Reduced row echelon form of [ A b ] solution of Ax = b. This is called the method of reduction. Example.3.7 (Reduced row echelon form) The following are all the possible reduced row echelon forms of 2 2 matrices. [ ], [ ], [ ], [ ]. The following are all the possible reduced row echelon forms of 3 3 matrices.,,,,,,. Example.3.8 (Reduced row echelon form) The following are the reduced row echelon forms of the matrices in Examples.3.2 and.3.3 (page ), respectively.,. 4
.3 Theory of Solutions Existence and Uniqueness In solving a specific system Ax = b of linear equations, we have to first clarify Existence: Does the system have solutions? If the answer is no, then the system has no solution. If the answer is yes, then we may further clarify Uniqueness: Does the system have a single or more than one solutions? If the solution is single, then the system has a unique solution. If the solution is many, then the system has non-unique solutions. Thus a basic question about a system Ax = b is which of the three possibilities (no solution, unique solution, non-unique solutions) is the solution in. The answer to the question can be answered from the row echelon form of the augmented matrix [ A b ]. Furthermore, the solution (if any) of the system is immediately given by the reduced row echelon form of [ A b ]. Example.3.9 (Row echelon form vs. existence and uniqueness) Let us recall again the row echelon forms of the systems of three linear equations with three variables in Examples.2.3,.2.4 and.2.5 (page 8). They are respectively, 4 2, 3 3 2. The corresponding reduced row echelon forms are 6, 4 2, 2 2. (.4) The corresponding systems of linear equations and their solutions are x = 6, x 2 =, x 3 = 4 x +2x 3 =, x 2 x 3 =, = x +x 3 = 2, x 2 2x 3 =, = = unique solution : x = 6, x 2 =, x 3 = 4, = no solution ( = is a contradiction), = non-unique solutions : x = r + 2, x 2 = 2r, x 3 = r. In general, given a system Ax = b of linear equations, we first find the row echelon form of the augmented matrix [ A b ] [ ], from which we can determine which columns of A b are pivot and which are not. 5
. Systems of Linear Equations Theorem.3. The existence and uniqueness of the solutions of Ax = b is determined by pivot columns of [ A b ] as follows. Pivot columns of [ A b ]. b pivot no solution 2. b not pivot, all A-columns pivot unique solution Existence and uniqueness for Ax = b 3. b not pivot, not all A-columns pivot non-unique solutions Let us explain in more details of Example.3.9 based on the above theorem. By (.4) we circle the pivots in the reduced row echelon forms of the corresponding augmented matrices: 6 2 2,, 2. 4 For the first matrix of the above, since the last column (i.e., b-column) is not a pivot column, the corresponding system has solutions. Meanwhile, since the first, second, and third columns (i.e., all the A-columns) are pivot columns, the solution is unique. Therefore the system has a single solution. For the second matrix of the above, since the last column (i.e., b-column) is a pivot column, the system has no solution. For the third matrix of the above, since the last column (i.e., b-column) is not a pivot column and the third column is not a pivot column (i.e., not all A-columns are pivot columns), the system has non-unique solutions. Example.3. (Determine existence and uniqueness) For which b, b 2, b 3 that the following system of linear equations has solutions? x +4x 2 = b, 2x +5x 2 = b 2, 3x +6x 2 = b 3. Solution We do the row operations, 4 b 4 b 2 5 b 2 2R +R 2 3 b 2 2b 3 6 b 3 3R +R 3 6 b 3 3b 2R 2+R 3 4 b 3 b 2 2b. b 2b 2 + b 3 Thus we conclude that the system has solutions the last column is not pivot b 2b 2 + b 3 =. Example.3. (Determine existence and uniqueness) For which values of a that the following system of linear equations x +x 2 x 3 = 2, x ax 3 =, x +ax 2 = has no solution, unique solution, non-unique solutions? 6
.4 Homogeneous Systems Solution We do the row operations, 2 2 a R +R 2 a a R +R 3 a (a ) R 2+R 3 2 a. 2a a 2 a Thus we conclude that if. 2a a 2 ( a, 2), then we have 2 2 a a. a 2a a 2 a This shows that all except the last column are pivot and hence the system has a unique solution. 2. a =, then we have 2 a 2. a This shows that the third and the last columns are not pivot and hence the system has non-unique solutions. 3. a = 2, then we have 2 a 2. a 2 This shows that the last column is pivot and hence the system has no solution..4 Homogeneous Systems A linear system is called homogeneous if all the constant terms are zero, like x + 2x 2 x 3 =, 2x x 2 + 5x 3 =, 3x + 3x 2 x 3 =. Using matrix-vector notation, we can rewrite the above homogeneous linear system as 2 x Ax =, where A = 2 5, x = x 2, =. 3 3 x 3 In general, when A is an m n coefficient matrix, then x will be a vector of size n, and will be a vector of size m (called the zero vector of size m). We will use the same notation for zero vectors of different sizes, as the actual size of the vector is usually quite clear from the context. A homogeneous system is always consistent, as one can easily find a solution for it, namely x = x 2 = =. 7
. Systems of Linear Equations Such a solution x = (note that is now an n-vector) will be called the zero solution. Because it is trivial to find this solution, we sometimes call it the trivial solution (for the homogeneous system). Therefore, given a homogeneous system, we are interested in whether it has a non-trivial solution, i.e., a solution in which not all the variables are zeros. Recall that the solution set of a linear system has three possible cases: (i) No solution exists: This case will not happen for a homogeneous system. (ii) Has a unique solution: This case happens when all the variables are basic variables, and this unique solution must be the zero solution. (iii) Has infinitely many solution: This case happens when there are at least one free variable(s). So, non-trivial solutions exist only in case (iii). Recall that this will happen when the rank of the coefficient matrix A is smaller than the total number of variables. In particular, it is always the case in the following theorem. Theorem.4. If a homogeneous system Ax = has more unknowns than the equations, it has nontrivial solutions. Proof: Let the size of the coefficient matrix A be m n (m rows and n columns), where m < n. Recall the fact that we cannot have two pivot positions sitting in the same row, so A has at most m pivot positions, i.e., rank A m. Thus, So there will be non-trivial solutions. number of free variables = n rank A n m >. Remark. In case m n, no corresponding results available (may or may not have non-trivial solutions). One need to check directly the existence of free variables instead. Fundamental solutions of a homogeneous system The augmented matrix of a homogeneous system is of the form [ A ]. Performing row operations to the augmented matrix thus will not change the constant terms, so the resulting new augmented matrix still represents a homogeneous system. In the RREF, we can solve the system by assigning free parameters to the free variables, and then express the basic variables in terms of the free parameters. Let us consider a concrete example. Example.4. (Homogeneous system) Consider the following homogeneous system: x +x 2 2x 3 +3x 4 +4x 5 =, 2x +x 2 3x 3 +8x 4 +5x 5 =, x +x 2 2x 3 +2x 4 +2x 5 =, 3x +2x 2 5x 3 +x 4 +7x 5 =. 8
.4 Homogeneous Systems Solution First we transform the augmented matrix to its RREF. 2 3 4 [ ] A = 2 3 8 5 2 2 2 3 2 5 7 2 3 4 R 2 +R 4 2 3 2 2 5 R 2 +R 2 3 R 2 2 2R +R 2, R +R 3 3R +R 4 R 3 +R 4 R 3 5R 3 +R 2R 3 +R 2 2 3 4 2 3 2 5 2 3 4 2 3 2 9 7 2. Set x 3 = s, x 5 = t, then the solution of the system Ax = can be represented as x = s + 9t, x 2 = s 7t, x 3 = s, x 4 = 2t, x 5 = t. We may also write the solution in vector form: x 9 x 2 x = x 3 x 4 = s + t 7, where s, t R. 2 x 5 The variables can be expressed by using s, t, only as the constant terms are all zeros. Here we observe two specific solutions: 9 7 x = (choose s =, t = ), x2 = (choose s =, t = ). 2 We will call them fundamental solutions of the homogeneous system, as all the other solution of the system can be expressed in terms of them. The results of the above example can be generalized to the following theorem. Theorem.4.2 Let Ax = be a homogeneous system with k free variables. Set s, s 2,, s k the corresponding free parameters. Then any solution of Ax = can be expressed as to be x = s x + s 2x 2 + + s k x k, where x i is the solution vector associated to the free parameter s i (i.e., set s i = and all other s j = ). 9
. Systems of Linear Equations 2 Definition The above solution vectors x, x 2,, x k are said to form a set of fundamental solutions of the linear system produced by row reduction algorithm. In case Ax = has no free variable, we say that the system has no fundamental solution (but still have the unique zero solution). 3 Definition An expression like s x + s 2x 2 + + s k x k (where s, s 2,, s k are numbers) is called a linear combination of x, x 2,, x k. Example.4.2 (Fundamental solutions) Find the fundamental solutions, if any, of the homogeneous systems with the following coefficient matrices. [ ] 2 2 2 (i), (ii) 2 5, (iii) 2 2 2 3. 3 Solution We first note that in Example.4. we have applied the row operations to the augmented matrix. If you read the computations carefully, you will see the right side zero vector indeed has no role in the analysis. So we will have the same conclusion if we perform the row operations to the coefficient matrix only. For this example (and from now on), since the constant terms of a homogeneous system are all zeros, we can work directly on the coefficient matrix instead of the augmented matrix (but keep in mind that the last column now corresponds to a variable). For (i), [ ] 2 2 2 3 [ 2 7 ] [ ]. Therefore, x, x 3 are basic variables, x 2 is a free variable. Set x 2 = s, then x s x 2 = s = s, where s R. x 3 So, there is one fundamental solution of the homogeneous system, which is the vector (,, ). For (ii), 2 2 2 2 5 5 3 5 3. 3 5 4 7 We observe that all the variables of the system are basic variables. So there will be no fundamental solution. For (iii), the coefficient matrix is already in RREF. Therefore, x, x 3, x 5 are basic variables, and x 2, x 4, x 6 are free variables. Set x 2 = s, x 4 = t, x 6 = u, then x 2s t 2 x 2 s x 3 x 4 = 2t t = s + t 2 + u, where s, t, u R. x 5 x 6 u The above three vectors form a set of fundamental solutions of the homogeneous system. 2