MTH 3 Linear Algebra Study Guide Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education June 3,
ii
Contents Table of Contents iii Matrix Algebra. Real Life Examples...................................... Matrix Addition and Scalar Multiplication........................ 3.3 Matrix Multiplication................................... 4.4 Special Matrices...................................... 7.5 Partitioned Matrices.................................... 8.6 Inverse of a Matrix..................................... 9.7 Matrix Invertibility......................................8 Solve Matrix Equations by Inverse............................ 3.9 Rank of a Matrix...................................... 7 Matrix Algebra (True or False) 9 Matrix Algebra (Worked Examples) 3 Linear Equations 4. Systems of m Equations and n Unknowns........................ 4. Solve Systems of Linear Equations............................ 43.3 Geometrical Interpretation of Solutions......................... 44.4 Gaussian Elimination.................................... 48.5 (Reduced) Row Echelon Form............................... 55.6 Elementary Matrices.................................... 58.7 Existence and Uniqueness................................. 58.8 Homogeneous Systems................................... 6.9 Fundamental solutions................................... 6 Linear Equations (True or False) 65 Linear Equations (Examples done in Class) 7 3 Vector Spaces 5 3. Geometry of Euclidean vectors in R, R 3......................... 5 3. Vector Addition and Scalar Multiplication........................ 6 3.3 Linear Combination of Vectors.............................. 7 3.4 Express Solution in Vector Form............................. 8 3.5 Find General Solution from RREF............................ 8 3.6 Free Variables and Rank.................................. 3 iii
CONTENTS 3.7 Vector Spaces and Subspaces............................... 3 3.8 Column Space, Row Space and Null Space........................ 36 3.9 Span of Vectors....................................... 37 3. Linear Independence.................................... 4 3. Basis of Euclidean Spaces (R n ).............................. 45 3. Basis of Subspaces (Column Space, Row Space, Null Space).............. 49 3 Vector Spaces (True or False) 5 3 Vector Spaces (Worked Examples) 57 4 Determinants and Eigenvalues 79 4. Find Determinants by Cofactor Expansions....................... 79 4. Determinants and Inverses................................. 79 4.3 Definitions of Egenvalue and Eigenvector......................... 8 4.4 Characteristic Equation.................................. 83 4.5 Diagonalization....................................... 87 4.6 Diagonalizability...................................... 88 4.7 Application of Diagonalization.............................. 9 4 Determinants and Eigenvalues (True or False) 93 4 Determinants and Eigenvalues (Worked Examples) 97 5 Vector Geometry 9 5. Geometric Vectors..................................... 9 5. Dot Product and Inner Product.............................. 9 5.3 Line and Plane....................................... 5.4 Orthogonal and Orthonormal Set............................. 5.5 Orthogonality and Linear Independence......................... 3 5.6 Orthogonal Matrix..................................... 5 5.7 Orthogonal Projection................................... 6 5.8 Gram Schmidt Orthogonalization............................. 9 5.9 Projection and Approximation (Least Squares Applications.............. 3 5 Vector Geometry (True or False) 35 5 Vector Geometry (Worked Examples) 39 6 Miscellaneous 57 6. Cross Product........................................ 57 6. Linear Transformation................................... 58 6.3 Linear Operators and Similarity.............................. 59 6.4 Fourier Transformation................................... 6 7 Answers to True or False Questions 6 iv
Chapter Matrix Algebra Our ability to analyse and solve linear equations will be greatly enhanced when we can perform algebraic operations with matrices. Furthermore, the definitions and theorems in this chapter provide some basic tools for handling the many applications of linear algebra that involve two or more matrices.. Real Life Examples Matrices are a very important tool in expressing and discussing problems which arise from real life examples. Example. (Electric network) In the following electric network, 5Ω P Ω V + Ω V + i i i Ω 3 Q Figure 5.: An electric network the currents i, i, i 3 satisfy the equations Node P : i +i i 3 =, Node Q : i i +i 3 =, Left loop : 5i +i =, Right loop : i +3i 3 =. The above system of equations is characterized by the following rectangular array of numbers [ ] A b = 5, 3 called the augmented matrix.
. Matrix Algebra Example. (Rotation in plane) Consider the rotation by an angle θ in the plane with respect to the origin. The rotation maps a point v = (x, y) to another point v = ( x, ȳ). v = ( x, ȳ) θ v = (x, y) Figure 5.: Rotation by angle θ v may be expressed in terms of v by the formula { x = xcos θ y sin θ, ȳ = xsin θ + y cos θ. The formula is characterized by the following rectangular array of numbers [ ] cos θ sin θ R θ =, sin θ cos θ called the transformation matrix of the rotation. Example.3 (Spring-mass system) In the following spring-mass system (where m, m are masses, and k, k, k 3 are spring constants), k = 7 m = k = 5 m = 8 k 3 = 4 Figure 5.3: Spring-mass system the displacement functions x (t) and x (t) of the two masses satisfy the following equations of motions { x = 7x 5(x x ), x = 6 5 x + x, 8x = 4x 5(x x ), or x = The differential system is characterized by the following array of numbers 6 5 5 9, 8 8 called the coefficient matrix. 5 8 x 9 8 x.
Chapter Linear Equations. Systems of m Equations and n Unknowns In this chapter we first review how systems of linear equations involving two variables are solved using the method of substitution learn in secondary school elementary algebra. (Basic properties of linear equations and lines should be reviewed.) Because the method of substitution is not suitable for linear systems involving large numbers of equations and variables, we then turn to a different method of solution involving the concept of an augmented matrix, which arises quite naturally when dealing with larger linear systems. We then study matrices and matrix basic operations in their own right as a new mathematical tool. With these new operations added to our mathematical toolbox, we return to systems of linear equations from a fresh point of view. A new technique, called the elementary row operations, will play an important role. Solution of systems of linear equations has its own importance in applications and they appear frequently in many practical problems. We briefly review several real life problems in this section. To establish basic concepts, we consider the first simple example. Example. (Systems with two variables) If adult tickets and child ticket cost $8, and if adult ticket and 3 child tickets cost $9, what is the price of each? Solution Let x be the price of adult ticket and y the price of child ticket. Then x + y = 8 and x + 3y = 9. We now have a system of two linear equations with two variables. It is easy to find ordered pairs (x, y) that satisfy one or the other of these equations. For example, the ordered pair (4, ) satisfies the first equation, but not the second, and the ordered pair (6, ) satisfies the second, but not the first. To solve this system, we must find all ordered pairs of real numbers that satisfy both equations at the same time. The set of all such ordered pairs is called the solution set. The method of substitution works nicely for systems involving two variables. By eliminating one variable from these two equations will immediately solve the problem. For example, if we choose to eliminate y, then we may substitute y = 8 x (follows from the first equation) into the second equation. That gives x + 3(8 x) = 9. After solving, we have the solution x = 3 and hence y =. In the following we give another example: The quantity of a product that people are willing to buy during some period of time depends on its price. Generally, the higher the price, the less the demand; the lower the price, the greater the demand. Similarly, the quantity of a product that a supplier is willing to sell 4
. Linear Equations during some period of time also depends on the price. Generally, a supplier will be willing to supply more of a product at higher prices and less of a product at lower prices. The simplest supply and demand model is a linear model where the graphs of a demand equation and a supply equation are straight lines. Example. (Supply and demand) Suppose that we are interested in analyzing the sale of cherries each day in Hong Kong. Using special analytical techniques (regression analysis) and data collected, an analysis arrives at the following pricedemand and price-supply models: p = q + 4, (Price-demand equation, consumer) p =.7q + 7.6, (Price-supply equation, supplier) where q represents the quantity of cherries in thousands of pounds and p represents the price in dollars. For example, we see that consumers will purchase thousand pounds (q = ) when the price is p = () + 4 = dollars per pound. On the other hand, suppliers will be willing to supply 7.74 thousand pounds of cherries at dollars per pound (solve =.7q + 7.6). Thus, at $ per pound the suppliers are willing to supply more cherries than consumers are willing to purchase. The supply exceeds the demand at that price and the price will come down. At what price will cherries stabilize for the day? That is, at what price will supply equal demand? This price, if it exists, is called the equilibrium price, and the quantity sold at that price is called the equilibrium quantity. The result could also be interpreted geometrically. The point where the two straight lines (two curves in general) for the price-demand equation and the price-supply equation intersect is called the equilibrium point. How do we find these quantities? Solution We solve the linear system p = q + 4, (Demand equation) p =.7q + 7.6 (Supply equation) by using the method of substitution (substituting p = q + 4 into the second equation) q + 4 =.7q + 7.6,.7q = 3.4, q = thousand pounds. (equilibrium quantity) Now substitute q = back into either of the original equations in the system and solve for p (we choose the first equation) p = () + 4 = 6. dollars per pound. (equilibrium price) If the price is above the equilibrium price of $6. per pound, the supply will exceed the demand and the price will come down. If the price is below the equilibrium price of $6. per pound, the demand will exceed the supply and the price will rise. Thus, the price will reach equilibrium at $6.. At this price, suppliers will supply thousand pounds of cherries and consumers will purchase thousand pounds. 4
. Solve Systems of Linear Equations. Solve Systems of Linear Equations The substitution method used in the previous examples is an algebraic method that is easy to use and provides exact solutions to a system of two equations with two variables, provided that solutions exist. We now define some terms that we can use to describe the different types of solutions to systems of equations that we will encounter. Solutions may exist or may not exist. See the following definition. Definition A system of linear equations is called consistent if it has one or more solutions and inconsistent if no solutions exist. Furthermore, a consistent system is said to be independent if it has exactly one solution (often referred to the unique solution) and dependent if it has more than one solution (referred to infinitely many solutions). Two systems of linear equations are equivalent if they have the same solution set. Referring to the system in Example., the system is a consistent and independent system with exactly one solution x = 3, y =. A natural question may arise: Can a consistent system have exactly two solutions? Exactly three solutions? The answer is simply NO. For example, by geometrically interpreting a system of two linear equations with two variables, we gain useful information about what to expect in the way of solutions to the system. In general, any two lines in a coordinate plane must intersect in exactly one point, be parallel, or coincide (have identical graphs). In fact there are only three possible types of solutions for systems of two linear equations in two variables. These ideas are illustrated geometrically in the following. Given a system of linear equations, we will have one of the three possibilities for the number of solutions.. Exactly one solution (i.e., a unique solution).. No solution. 3. Infinitely many solutions (i.e., non-unique solutions). Let us first review what are solutions to systems of two linear equations with two variables and its geometrical interpretation. 43
. Linear Equations.3 Geometrical Interpretation of Solutions The pair x =, x = 3 is a solution of the following system of linear equations { x x =, x + x = because if we replace the variables by the given values, then the two equalities hold. Another pair x =, x = is not a solution because the first equality does not hold. x (, 3) x + x = (, ) x x x = Figure.: Unique solution The two linear equations represent two straight lines on the (two-dimensional) plane. The solution (x, x ) = (, 3) of the system of the two equations is the intersection of the two lines. In particular, we can find exactly one solution. The following system of linear equations { x x =, 4x x = represents two different parallel lines. Since the two lines never meet, the system does not have any solution. x 4x x = x x x = Figure.: No solution The following system of linear equations { x x =, 4x x = 44
.3 Geometrical Interpretation of Solutions represents two identical lines. Any point on the line is a solution to the system. In particular, the system has infinitely many solutions. x x x = 4x x = x Figure.3: Infinite number of solutions The geometrical interpretation of the solution of linear equations can be extended to more variables and more equations. For example, one linear equation with three variables a x + a x + a 3x 3 = b is represented by a plane in the (three-dimensional) space. A system of two linear equations with three variables is represented by two planes. The following are the possible intersections (= solution to the system) of the two planes. line of solutions no solution plane of solutions Figure.4: Intersection of two planes In Figure.4, we observed that a system of linear equations with 3 variables cannot have a unique solution. In that case, if the system is consistent, it must have infinitely many solutions. In the following we also give some examples of linear systems of more than two variables. Just remark that no professional knowledge in chemistry nor physics is needed for this elementary course MATH 3, the following two examples are only used for illustrating the wide applicability of linear systems to real life examples. Example.3 (Electric network) In the following electric network, 5Ω P Ω V + Ω V + i i i Ω 3 Q Figure.5: An electric network 45
. Linear Equations the currents i, i, i 3 satisfy the equations Node P : i +i i 3 =, Node Q : i i +i 3 =, Left loop : 5i +i =, Right loop : i +3i 3 =. Example.4 (Linear equations in four variables) In order to balance the following chemical reaction x CH 4 + x O y CO + y H O, the number of molecules x, x, y, y must satisfy x y =, 4x y =, x y y =. Remind you that there is no geometrical meaning to the system of more than three variables. Referring to the systems in Examples.3.4, the first system (four equations with three variables) is a consistent and independent system with the unique solution: i = 78/3, i = 5/3, i 3 = 37/3. The second system (three equations with four variables) is a consistent and dependent system with an infinite number of solutions. Remark that for these two examples the method of substitution seems to be not as efficient as for the system of two equations with two variables. Students may try it out. Indeed we compute the solutions by a more general and efficient method called the Gaussian elimination. This method will be discussed in Section.4 (page 48). But before we go into the details of this new method we need to borrow some terminologies in matrices and express the problem in another point of view. Let us recall that a linear equation is a x + a x + + a nx n = b. We call a, a,, a n the coefficients of the equation, and x, x,, x n the variables of the equation. A system of linear equations is a list of linear equations. For example, a system of four equations with three variables (we say a four by three system ) is a x + a x + a 3x 3 = b, a x + a x + a 3x 3 = b, a 3x + a 3x + a 33x 3 = b 3, a 4x + a 4x + a 43x 3 = b 4. The system is completely characterized (besides the notations for the variables) by the following rectangular array of numbers called the coefficient matrix a a a 3 A = a a a 3 a 3 a 3 a 33 a 4 a 4 a 43 (.) 46
.3 Geometrical Interpretation of Solutions and the right side b b = b b 3. b 4 The system also corresponds to the augmented matrix a a a 3 b [ ] A b = a a a 3 b a 3 a 3 a 33 b 3. a 4 a 4 a 43 b 4 We emphasize the following correspondence columns of A variables, rows of [ A b ] equations. (.) Example.5 (Augmented matrix) The augmented matrices of the systems in Examples.3 and.4 are respectively 5 and 4. 3 In terms of matrices, the system (.) is equivalent to simply Ax = b, where x = (x, x, x 3). Next we review some basic operations of matrices which will be useful to our later discussion of solving general system Ax = b by Gaussian elimination. In Chapter we will confine our attention to the world of matrices. Now, we will focus on the techniques of solving systems of linear equations. 47
. Linear Equations.4 Gaussian Elimination The method of substitution may work well for systems involving two variables. However, it is not easily extended to larger systems. In fact, the most efficient and practical way for solving systems of linear equations is through an algorithm called Gaussian elimination. The idea is to simplify the system by creating as many zeros as in the coefficients. The solution may then be easily deduced from the simplified system. The Gaussian elimination is probably the most important method of solution. It readily generalizes to larger systems and forms the basis for computer-based solution methods. As a matter of fact, all the important computer packages nowadays for solving systems of linear equations are based on Gaussian elimination. If you have learned before to use the Cramer s rule to solve systems of linear equations, then you should forget this old method as soon as possible. Cramer s rule is a limited method because of the following:. The computation of determinants is much more complicated than elimination.. Cramer s rule cannot solve underdetermined (such as equations, 3 variables) or overdetermined (such as 3 equations, variables) systems. Gaussian elimination has no such limitations and indeed it works for any linear systems of m equations with n variables, where m < n, m = n, m > n are all permissible. Now, with some knowledge of matrix notations and operations (details can be found in Chapter ) we are going to introduce the method of Gaussian elimination formally. But, first of all, we would also like to give you the insight of how and why the method works. Let s read the very simple example. Example.6 (Equivalent equations) To solve an equation such as x 5 = 3, we perform permissible operations on the equation until we reach an equivalent equation whose solution is obvious. x 5 = 3, x 5 + 5 = 3 + 5, x = 8, x = 8, x = 4. Recall that we added 5 to both sides in the second step and divided both sides by in the fourth step. These two operations simply convert the equation x 5 = 3 to an equivalent equation x = 4. The solution follows as we wish. The following theorem indicates that we can solve systems of linear equations in a similar manner. Theorem.4. (Three operations that produce equivalent systems) A system of linear equations is transformed into an equivalent system if (A) (B) (C) Two equations are interchanged. An equation is multiplied by a nonzero constant. A constant multiple of one equation is added to another equation. 48
.4 Gaussian Elimination Any one of the three operations in Theorem.4. can be used to produce an equivalent system. The basic principle here is to transform a given (complicated) system into an equivalent but simpler system through a series of operations. Based on this principle we have to eliminate as many variables as possible. A system Ax = b of linear equations is completely characterized by its augmented matrix [ A b ], as we mentioned in Section.3 (page 46). Gaussian elimination can then be considered as a manipulation on the augmented matrix. In the following examples, we indicate the relation by writing systems and augmented matrices side by side. Theorem.4. is used in the examples that rows of [ A b ] are equivalent to equations. Example.7 ( system with unique solution) Solve the system of linear equations { x x =, x +x =. Solution Let us solve the system { x x =, x +x =. [ ] We do the row operation R + R (add twice of second equation/row to the first equation/row) to obtain { x = 3, x +x =. [ ] 3 We may further do R R (exchange the first and the second equations/rows) so that the equations are listed from the most complicated to the simplest { x +x =, x = 3. [ ] 3 Substituting x = 3 into the first equation, we get x = x =. Example.8 (3 3 system with unique solution) Solve the system of linear equations x +x +x 3 =, x +3x +x 3 =, 3x +8x +x 3 =. Solution Let us solve the system x +x +x 3 =, x +3x +x 3 =, 3x +8x +x 3 =. We do R + R and 3R + R 3 to get x +x +x 3 =, x =, 5x x 3 =. 3 3 8 5 49
. Linear Equations Then we do 5R + R 3 to get x +x +x 3 =, x =, x 3 = 4. 4 This corresponds to x = and x 3 = 4. Substituting them into the first equation, we have x = 6. Example.9 (3 3 system without solution) Solve the system of linear equations x +x +x 3 =, x +3x x 3 = 4, x x 3 =. Solution We simplify the augmented matrix as follows 3 4 R +R R R +R 3. Interpreted as a system of linear equations, we have x +x +x 3 =, x x 3 =, =. The last equation is a contradiction which implies that the system has no solution. Example. (3 3 system with non-unique solutions) Solve the system of linear equations x x +3x 3 = 3, 3x +x +x 3 = 5, x x 3 =. Solution We simplify the augmented matrix as follows 3 3 3 3 3 5 3R +R 4 8 4 3 3 4R 3 +R R R 3 3 3. Interpreted as a system of linear equations, we have x x +3x 3 = 3, x x 3 =, =. 5
.4 Gaussian Elimination The last equation is redundant. From the second equation, we have x = x 3. Substituting this into the first equation, we have x = (x 3 ) 3x 3 + 3 = x 3 +. The system has non-unique solutions (x, x, x 3) = ( r +, r, r), for any real number r. 5
. Linear Equations To summarize, the Gaussian elimination is done through the following three operations. Table.: Three (row) operations used in the Gaussian elimination. notation rr i + R j rr i R i R j description adding r times the i-th equation/row to the j-th equation/row multiplying r to the i-th equation/row exchanging the i-th equation/row with the j-th equation/row When applied to matrices, the three operations are also called row operations. The three operations do not change the solutions of the system of equations. By making use of the three operations, the system may eventually be simplified to the following upper triangular shape.... Then one may solve the system by backward substitution (i.e., solving the system from bottom up, and finding the values of more and more variables). Example. (4 3 system with unique solution) Solve the system of linear equations x x +x 3 =, x +x x 3 =, x +4x = 5, x +3x 3 =. Solution Let us solve the system. We simplify the augmented matrix as follows 4 5 4 5 3 3 3 6 3 3. The last row is redundant while the third row implies that x 3 =. Substituting this value into the second row, we have x = 5/. Substituting the values of x and x 3 into the first row, we have x = /. Hence the system has the unique solution (x, x, x 3) = (, 5, ). 5
.4 Gaussian Elimination Having discussed how to simplify a system of linear equations, we may ask: how simple can a system be reduced to? The answer to the question can tell us a lot of information about the solutions of the system, particularly the existence and uniqueness of solutions. Row Echelon Form In Examples.7,.8,.9, and., the systems (or rather, the augmented matrices [ A b ] ) are reduced to the following shapes [ ],,,, (.3) where are nonzero numbers, and are any numbers. These are the simplest shapes you can get by the three row operations. These are called the row echelon forms of [ A b ]. Generally speaking, a matrix is called in row echelon form if the following two conditions hold (where a leading nonzero entry of a row of the matrix is the first nonzero element in the row counted from the left): () All zero rows, if any, are at the bottom of the matrix. () Each leading nonzero entry in a row is to the right of the leading nonzero entry in the preceding row. Example. (Row echelon form) The following are all the possible row echelon forms of matrices ( simplest shapes of linear equations with variable). [ ] [ ] [ ] [ ],,,. The following are all the possible row echelon forms of 3 3 matrices ( simplest shapes of 3 linear equations with variables).,,,,,,. In a row echelon form, the entry denoted by is located further and further to the right when one goes from the top to the bottom. We call these entries pivots of the matrix, and the columns containing the pivots the pivot columns of the matrix. Example.3 (Row echelon form) The following is the row echelon form for the augmented matrix in Example.9. The pivots are the (, )-, (, )-, and (3, 4)-entries. The pivot columns are the first, second, and the fourth columns. 53
. Linear Equations Example.4 (Row echelon form) The following is a more complicated row echelon form. The pivots are the (, )-, (, 4)-, (3, 5)-, and (4, 7)-entries. The pivot columns are the second, fourth, fifth, and the seventh columns. Example.5 (Non-row echelon form) The matrix is not in row echelon form because the for the second row is after the for the third row. If we exchange the second and the third rows (so that the matrix is further simplified), then we do get a row echelon form. Similarly, the matrix is not in row echelon form because the for the second row is not before the for the third row. Exchanging rows will never give us a row echelon form. We have to use row operations such as R 3 + R and R R 3 (or simply one operation R + R3) to further simplify the matrix in order to get a row echelon form. 54
.5 (Reduced) Row Echelon Form.5 (Reduced) Row Echelon Form Row echelon form is the simplest shape one can get by performing row operations. What is the ultimate simplest matrix one can get? Example.6 (Reduced row echelon form) The system in Example.8 (page 49) has been simplified to the following row echelon form. 4 We reduce the coefficient of (3, 3)-pivot to by doing R 3. 4 We cancel entries above the (, 3)-pivot by doing R 3 + R 5. 4 We cancel entries above the (, )-pivot by doing R + R 6. 4 Interpreted the above (reduced) row echelon form as a system of linear equations, we have x = 6, x =, x 3 = 4, which immediately gives the unique solution of the original system. Example.7 (Reduced row echelon form) The system in Example. has been simplified to the following row echelon form 3 3. We use the (, )-pivot to cancel entries above it by doing R + R. Interpreted the above (reduced) row echelon form as a system of linear equations, we have x +x 3 =, x x 3 =, =, which gives the non-unique solutions of the system x = r +, x = r, x 3 = r, with r arbitrary. 55
. Linear Equations In similar ways, we may further reduce the row echelon forms (.3) to the following [ ],,,, where are reduced to, the entries above are reduced to, and are again any numbers. These are the simplest matrices you can get by the three row operations, and we call them the reduced row echelon forms of [ A b ]. 56
.5 (Reduced) Row Echelon Form Generally speaking, a matrix is called in reduced row echelon form if it is in row echelon form, that is, if it satisfies the two conditions () and () (on page 53), and if it satisfies the following additional two conditions: (3) Each pivot (leading nonzero entry) is equal to. (4) Each pivot is the only nonzero entry in its column. By comparing Examples.6,.7 with Examples.8,., we observe that the advantage of reduced row echelon form is that we can write down the solutions directly, without any backward substitution. Reduced row echelon form of [ A b ] solution of Ax = b. This is called the method of reduction. Example.8 (Reduced row echelon form) The following are all the possible reduced row echelon forms of matrices. [ ] [ ] [ ] [ ],,,. The following are all the possible reduced row echelon forms of 3 3 matrices.,,,,,,. Example.9 (Reduced row echelon form) The following are the reduced row echelon forms of the matrices in Examples.3 and.4 (page 53), respectively.,. 57
. Linear Equations.6 Elementary Matrices.7 Existence and Uniqueness In solving a specific system Ax = b of linear equations, we have to first clarify Existence: Does the system have solutions? If the answer is no, then the system has no solution. If the answer is yes, then we may further clarify Uniqueness: Does the system have a single or more than one solutions? If the solution is single, then the system has a unique solution. If the solution is many, then the system has non-unique solutions. Thus a basic question about a system Ax = b is which of the three possibilities (no solution, unique solution, non-unique solutions) is the solution in. The answer to the question can be answered from the row echelon form of the augmented matrix [ A b ]. Furthermore, the solution (if any) of the system is immediately given by the reduced row echelon form of [ A b ]. Example. (Row echelon form vs. existence and uniqueness) Let us recall again the row echelon forms of the systems of three linear equations with three variables in Examples.8,.9 and. (page 49). They are respectively, 4, 3 3. The corresponding reduced row echelon forms are 6, 4,. (.4) The corresponding systems of linear equations and their solutions are x = 6, x =, x 3 = 4 x +x 3 =, x x 3 =, = x +x 3 =, x x 3 =, = = unique solution : x = 6, x =, x 3 = 4, = no solution ( = is a contradiction), = non-unique solutions : x = r +, x = r, x 3 = r. 58
.7 Existence and Uniqueness In general, given a system Ax = b of linear equations, we first find the row echelon form of the augmented matrix [ A b ] [ ], from which we can determine which columns of A b are pivot and which are not. Theorem.7. The existence and uniqueness of the solutions of Ax = b is determined by pivot columns of [ A b ] as follows. Pivot columns of [ A b ]. b pivot no solution. b not pivot, all A-columns pivot unique solution Existence and uniqueness for Ax = b 3. b not pivot, not all A-columns pivot non-unique solutions Let us explain in more details of Example. based on the above theorem. By (.4) we circle the pivots in the reduced row echelon forms of the corresponding augmented matrices: 6, 4,. For the first matrix of the above, since the last column (i.e., b-column) is not a pivot column, the corresponding system has solutions. Meanwhile, since the first, second, and third columns (i.e., all the A-columns) are pivot columns, the solution is unique. Therefore the system has a single solution. For the second matrix of the above, since the last column (i.e., b-column) is a pivot column, the system has no solution. For the third matrix of the above, since the last column (i.e., b-column) is not a pivot column and the third column is not a pivot column (i.e., not all A-columns are pivot columns), the system has non-unique solutions. Example. (Determine existence and uniqueness) For which b, b, b 3 that the following system of linear equations has solutions? x +4x = b, x +5x = b, 3x +6x = b 3. Solution We do the row operations, 4 b 5 b 3 6 b 3 R +R 3R +R 3 4 b 3 b b 6 b 3 3b R +R 3 4 b 3 b b. b b + b 3 Thus we conclude that the system has solutions the last column is not pivot b b + b 3 =. Example. (Determine existence and uniqueness) 59
. Linear Equations For which values of a that the following system of linear equations x +x x 3 =, x ax 3 =, x +ax = has no solution, unique solution, non-unique solutions? Solution We do the row operations, a R +R a a R +R 3 a (a ) R +R 3 a. a a a Thus we conclude that if. a a ( a, ), then we have a a. a a a a This shows that all except the last column are pivot and hence the system has a unique solution.. a =, then we have a. a This shows that the third and the last columns are not pivot and hence the system has non-unique solutions. 3. a =, then we have a. a This shows that the last column is pivot and hence the system has no solution. 6
.8 Homogeneous Systems.8 Homogeneous Systems A linear system is called homogeneous if all the constant terms are zero, like x + x x 3 =, x x + 5x 3 =, 3x + 3x x 3 =. Using matrix-vector notation, we can rewrite the above homogeneous linear system as x Ax =, where A = 5, x = x, =. 3 3 x 3 In general, when A is an m n coefficient matrix, then x will be a vector of size n, and will be a vector of size m (called the zero vector of size m). We will use the same notation for zero vectors of different sizes, as the actual size of the vector is usually quite clear from the context. A homogeneous system is always consistent, as one can easily find a solution for it, namely x = x = =. Such a solution x = (note that is now an n-vector) will be called the zero solution. Because it is trivial to find this solution, we sometimes call it the trivial solution (for the homogeneous system). Therefore, given a homogeneous system, we are interested in whether it has a non-trivial solution, i.e., a solution in which not all the variables are zeros. Recall that the solution set of a linear system has three possible cases: (i) No solution exists: This case will not happen for a homogeneous system. (ii) Has a unique solution: This case happens when all the variables are basic variables, and this unique solution must be the zero solution. (iii) Has infinitely many solution: This case happens when there are at least one free variable(s). So, non-trivial solutions exist only in case (iii). Recall that this will happen when the rank of the coefficient matrix A is smaller than the total number of variables. In particular, it is always the case in the following theorem. Theorem.8. If a homogeneous system Ax = has more unknowns than the equations, it has nontrivial solutions. Proof: Let the size of the coefficient matrix A be m n (m rows and n columns), where m < n. Recall the fact that we cannot have two pivot positions sitting in the same row, so A has at most m pivot positions, i.e., rank A m. Thus, So there will be non-trivial solutions. number of free variables = n rank A n m >. Remark. In case m n, no corresponding results available (may or may not have non-trivial solutions). One need to check directly the existence of free variables instead. 6
. Linear Equations.9 Fundamental solutions The augmented matrix of a homogeneous system is of the form [ A ]. Performing row operations to the augmented matrix thus will not change the constant terms, so the resulting new augmented matrix still represents a homogeneous system. In the RREF, we can solve the system by assigning free parameters to the free variables, and then express the basic variables in terms of the free parameters. Let us consider a concrete example. Example.3 (Homogeneous system) Consider the following homogeneous system: x +x x 3 +3x 4 +4x 5 =, x +x 3x 3 +8x 4 +5x 5 =, x +x x 3 +x 4 +x 5 =, 3x +x 5x 3 +x 4 +7x 5 =. Solution First we transform the augmented matrix to its RREF. 3 4 3 4 [ ] A = 3 8 5 R +R, R +R 3 3 3R +R 4 3 5 7 5 3 4 3 4 R +R 4 3 R 3 +R 4 3 R 3 5 R +R 3 9 5R 3 +R R 7 R 3 +R. Set x 3 = s, x 5 = t, then the solution of the system Ax = can be represented as x = s + 9t, x = s 7t, x 3 = s, x 4 = t, x 5 = t. We may also write the solution in vector form: x 9 x x = x 3 x 4 = s + t 7, where s, t R. x 5 The variables can be expressed by using s, t, only as the constant terms are all zeros. Here we observe two specific solutions: 9 7 x = (choose s =, t = ), x = (choose s =, t = ). 6
.9 Fundamental solutions We will call them fundamental solutions of the homogeneous system, as all the other solution of the system can be expressed in terms of them. The results of the above example can be generalized to the following theorem. Theorem.9. Let Ax = be a homogeneous system with k free variables. Set s, s,, s k the corresponding free parameters. Then any solution of Ax = can be expressed as to be x = s x + s x + + s k x k, where x i is the solution vector associated to the free parameter s i (i.e., set s i = and all other s j = ). 3 Definition The above solution vectors x, x,, x k are said to form a set of fundamental solutions of the linear system produced by row reduction algorithm. In case Ax = has no free variable, we say that the system has no fundamental solution (but still have the unique zero solution). 4 Definition An expression like s x + s x + + s k x k (where s, s,, s k are numbers) is called a linear combination of x, x,, x k. Example.4 (Fundamental solutions) Find the fundamental solutions, if any, of the homogeneous systems with the following coefficient matrices. [ ] (i), (ii) 5, (iii) 3. 3 Solution We first note that in Example.3 we have applied the row operations to the augmented matrix. If you read the computations carefully, you will see the right side zero vector indeed has no role in the analysis. So we will have the same conclusion if we perform the row operations to the coefficient matrix only. For this example (and from now on), since the constant terms of a homogeneous system are all zeros, we can work directly on the coefficient matrix instead of the augmented matrix (but keep in mind that the last column now corresponds to a variable). For (i), [ ] 3 [ 7 ] [ ]. Therefore, x, x 3 are basic variables, x is a free variable. Set x = s, then x s x = s = s, where s R. x 3 So, there is one fundamental solution of the homogeneous system, which is the vector (,, ). For (ii), 5 5 3 5 3. 3 5 4 7 63
. Linear Equations We observe that all the variables of the system are basic variables. So there will be no fundamental solution. For (iii), the coefficient matrix is already in RREF. Therefore, x, x 3, x 5 are basic variables, and x, x 4, x 6 are free variables. Set x = s, x 4 = t, x 6 = u, then x x x 3 x 4 x 5 x 6 = s t s t t u = s + t + u, where s, t, u R. The above three vectors form a set of fundamental solutions of the homogeneous system. 64
Chapter Linear Equations (True or False). If A is a square matrix and A = I, then A = I or A = I.. If AB = O, then A = O or B = O..3 If A, B, C are square and ABC = O, then one of them is O..4 If AB = AC, then B = C..5 If A is nonzero and AB = AC, then B = C..6 The square of a nonzero square matrix must be a nonzero matrix..7 If AB = BA, then (A + B) 3 = A 3 + 3A B + 3AB + B 3..8 An invertible matrix must be a square matrix..9 A non-square matrix can never be invertible.. If A has a zero row or a zero column, then A is not invertible.. If A is a square matrix which has no zero rows, then A is invertible. 65
. Linear Equations (True or False). Let A, B be invertible matrices of same size. Then AB is also invertible..3 Let A, B be invertible matrices of same size. Then A + B is also invertible..4 If AB is equal to the identity matrix, then A must be an invertible matrix..5 A, B are square matrices. If AB = I, then BA = I. Hence, A is invertible..6 For square matrix A, AA t = I if and only if A t A = I..7 If AB is invertible, then BA is invertible..8 If A O, then A is invertible..9 If A is invertible, then A O.. If A is a square matrix and A + 7A I = O, then A is invertible.. A symmetric matrix must be a square matrix.. If A is symmetric, so are A (if exists) and A 3..3 If B = A t A, then B is symmetric..4 If A is symmetric, so is f(a), for any polynomial f(x)..5 det (A + B) = deta + detb..6 det (ka) = k deta, for any integer k. 66
.7 deta t = ( ) deta..8 Three elementary row operations do not change the determinant of a square matrix..9 A row replacement operation does not change the determinant of a matrix..3 If A is row equivalent to B, then deta = detb..3 If A is row equivalent to B, then deta and detb are either both zero or both nonzero..3 The determinant of a triangular matrix is the sum of the entries on the main diagonal..33 Let A be a square matrix without zero rows and columns. Then A must be row equivalent to the identity matrix of same size..34 If A, B are nonzero square matrices and A is row equivalent to B, then both A, B are invertible..35 If A is an invertible matrix and B is row equivalent to A, then B is also invertible..36 rank (ca) = rank A, for any scalar c..37 If A is m n matrix with m < n, then rank A = m..38 If A is m n matrix with m > n, then rank A = m..39 An n n matrix having rank n is invertible..4 If m < n, then the system A m nx = always has a nontrivial solution..4 If A is an m n matrix with m > n, then Ax = always has a nontrivial solution. 67
. Linear Equations (True or False).4 If rank A m n = m with m < n, then Ax = b has solutions for all b..43 If rank A m n = m with m < n, then Ax = has only trivial solution..44 If ranka 7 5 = 5, then the solution of Ax = is unique..45 If ranka 7 5 = 5, then Ax = b has solutions for all b..46 If A m nx = b, with m < n, has solutions for all b, then rank A = m..47 If A m nx = b, with m < n, has solutions for all b, then rank A = n..48 If A m nx = b, with m < n, has solutions for all b, then A t x = has only trivial solution..49 If A m nx = b has solutions for all b R m, then m n..5 If A m nx = has only trivial solution, then m n..5 If A m nx = b has solutions for all b R m, then the rank of A is m..5 If A m nx = b has solutions for all b R m, then the rank of A is n..53 If A m nx = has only zero solution, then the rank of A is m..54 If A 3 5x = b has solutions for all b R 3, then A t x = has only trivial solution..55 If A and B have the same rank, then after a finitely many elementary row or column operations we can turn A into B..56 Let A be an n n matrix without zero rows and columns. If the equation Ax = b has a solution for a nonzero vector b R n, then the solution is unique. 68
.57 Let A be an n n matrix. If the equation Ax = b has a unique solution for a given nonzero vector b, then deta..58 Let A be an m n matrix. If the equation Ax = b has a unique solution for a nonzero vector b R m, then the homogeneous equation Ax = has only trivial solution..59 Let A be an m n matrix. The equation Ax = has only trivial solution if A has a pivot position on each row..6 Let A be an m n matrix. If Ax = has only trivial solution, then the rank of A is n..6 Let A be an n n matrix. The equation Ax = has a nontrivial solution if and only if deta =..6 Let A be an n n matrix. If the equation Ax = has only trivial solution, then A is row equivalent to the n n identity matrix..63 Let A be an n n matrix. The equation Ax = has only trivial solution if and only if det A..64 Let A be an 3 5 matrix. If the general solution of Ax = has two free parameters, then the rank of A is..65 If A is an 7 matrix, and the general solution of Ax = has 8 free variables, then ranka = 9..66 If Ax = b has solutions for all b, then the rank of A is the number of columns of A..67 If Ax = b has solutions for all b, then the rank of A is the number of rows of A..68 Suppose A is a non-square matrix and Ax = b has solutions for all b. Then, the number of rows of A must be larger than the number of columns of A..69 If rank [ A b ] > ranka, then Ax = b has no solution. 69
. Linear Equations (True or False).7 Let A be a 3 4 matrix. If the first, second, fifth columns of [ A b ] are pivot, then Ax = b has non-unique solutions..7 Let A be a square matrix. Then Ax = b has solutions for all b if Ax = has only trivial solution..7 The equation Ax = has only trivial solution if A is an m n matrix with m > n and does not contain a zero row..73 Let A be a square matrix. Then Ax = b has a unique solution if Ax = has only trivial solution..74 Let A be a 4 3 matrix. If [ A b ] is invertible, then Ax = b has no solution..75 Let A be a square matrix. If Ax = b has no solution for some b, then Ax = has a nontrivial solution..76 The equation Ax = b is consistent if the augmented matrix [ A b ] has a pivot position in every row..77 Let A be an m n matrix, B an n m matrix. Suppose n < m. One can always find A and B such that AB is invertible. 7
Chapter Linear Equations (Examples done in Class) Example. (Augmented matrix) Write down the augmented matrix for the system of linear equations. x x 3 = 5, x +x +x 4 =, x 3x 4 =. Suggested Solution 7
. Linear Equations (Examples done in Class) Example. (Augmented matrix) Suppose ax + by = represents a straight line which passes through the points (, ) and (3, 4). Write down the augmented matrix for the corresponding system of linear equations for a, b. Suggested Solution 7
Example.3 (Row echelon form) Which of the following matrices are in row echelon form? [ ] 3 3 4 A =, B =, C = 3, P =, Q =, R =, [ ] [ ] X =, Y =, Z =. Suggested Solution 73
. Linear Equations (Examples done in Class) Example.4 (Gaussian elimination unique solution) Solve the system of linear equations x +x x 3 =, x +3x x 3 =, 3x x 3 = 4. Suggested Solution 74
Example.5 (Gaussian elimination unique solution) Solve the system of linear equations x +x = 7, x 3x = 7, 3x 5x =. Suggested Solution 75
. Linear Equations (Examples done in Class) Example.6 (Gaussian elimination no solution) Solve the system of linear equations { x +y 3z =, x 4y +6z =. Suggested Solution 76
Example.7 (Gaussian elimination no solution) Solve the system of linear equations x x =, 3x x +x 3 = 8, x +x +x 3 = 3, x 3x x 3 =. Suggested Solution 77
. Linear Equations (Examples done in Class) Example.8 (Gaussian elimination non-unique solutions) Solve the system of linear equations { x x +3x 3 =, x 4x +3x 3 =. Suggested Solution 78
Example.9 (Gaussian elimination non-unique solutions) Solve the system of linear equations x x +x 3 =, 3x 6x +7x 3 =, x 4x +3x 3 =. Suggested Solution 79
. Linear Equations (Examples done in Class) Example. (Reduced row echelon form) Determine whether the matrices are in reduced row echelon form [ ] 3 A =, B =, C =, D =. Suggested Solution 8
Example. (Reduced row echelon form) Find the reduced row echelon form of the matrix 6 3 4. 3 Suggested Solution 8
. Linear Equations (Examples done in Class) Example. (Reduced row echelon form) Find the reduced row echelon form of the matrix 4 5. 5 5 5 Remark In the following Examples.3., we shall use the method of reduction to solve systems of linear equations. We should emphasize that the method of reduction is just the Gaussian elimination plus some more row eliminations until the augmented matrix is in its reduced row echelon form. The advantage of reduced row echelon form is that we can write down the solutions directly with no backward substitution is needed. Suggested Solution 8
Example.3 (Method of reduction unique solution) Solve the system of linear equations x +x 3 =, x +x x 3 = 3, x +x +4x 3 =. Suggested Solution 83
. Linear Equations (Examples done in Class) Example.4 (Method of reduction unique solution) Solve the system of linear equations x x =, x +x =, x 3x = 3. Suggested Solution 84
Example.5 (Method of reduction unique solution) Solve the system of linear equations x +y z =, x 4z = 8, 3x +y +z =, x y z = 4. Suggested Solution 85
. Linear Equations (Examples done in Class) Example.6 (Method of reduction unique solutions) Solve the system of linear equations x +y +z = 4a, x +y +z = 4b, x +y +z = 4c. Suggested Solution 86
Example.7 (Method of reduction non-unique solutions) Solve the system of linear equations x x +x 3 =, x +x 3 =, x 3x +4x 3 =. Suggested Solution 87
. Linear Equations (Examples done in Class) Example.8 (Method of reduction non-unique solutions) Solve the system of linear equations x +x +3x 3 +4x 4 = 5, x +3x +4x 3 +5x 4 = 6, 3x +4x +5x 3 +6x 4 = 7. Suggested Solution 88
Example.9 (Method of reduction non-unique solutions) Solve the system of linear equations x +x +5x 4 =, x +x 3 +x 4 =, x 3x +4x 3 7x 4 =, x x 3 +3x 4 =. Suggested Solution 89
. Linear Equations (Examples done in Class) Example. (Method of reduction non-unique solutions) Solve the system of linear equations x +x 3 +x 4 +4x 5 =, x +x 3 3x 4 =, 4x 3x +5x 3 +3x 4 +6x 5 =, x +x +4x 3 5x 4 +4x 5 = 3. Suggested Solution 9
Example. (Existence and uniqueness) Find the value of t for which the following system has solution and solve the system for this value of t by the method of reduction x +y =, tx +y = t, ( + t) x +y = 3. Suggested Solution 9
. Linear Equations (Examples done in Class) Example. (Existence and uniqueness) Find the value of h for which the following system has solution and solve the system for this value of h by the method of reduction x +x +x 3 =, x +3x +x 3 = 5, x +x +(h 5) x 3 = h. Suggested Solution 9
Example.3 (Existence and uniqueness) For which values of a and b does the system has () no solution, () a unique solution, (3) infinitely many solutions? x y +3z = 4, x 3y +az = 5, 3x 4y +5z = b. Suggested Solution 93
. Linear Equations (Examples done in Class) Example.4 (Existence and uniqueness) For which values of a does the system has () no solution, () a unique solution, (3) infinitely many solutions? x +y 3z = 4, x +y z =, 4x +5y +(a ) z = a + 6. Suggested Solution 94