System of Linear Equations

Transcription

1 CHAPTER 2 System of Linear Equations. Introduction Study of a linear system of equations is classical. First let s consider a system having only one equation: 2x+3y +4z = 5 (2.) Indeterminates x,y,z are referred to as unknowns of the equation. Both (,,0) and (,,) satisfy Equation (2.). In fact, assign whatever real numbers you like to x and y and find z from (2.) by substituting the values of x and y (you have assigned). Geometrically, the collection of all the solutions of Equation (2.) is a plane in the (real) space R 3. Suppose a,b,c,d R are fixed. Then, the collection of all (α,γ,β) R 3 satisfying equation ax+by +cz = d (2.2) is geometrically a plane in the space R 3 if at least one of a,b,c is nonzero. More generally, suppose, a,a 2,...,a n,b R. Then, the collection of all (α,α 2,...,α n ) R n satisfying n a i x i = b (2.3) i= is called a hyper plane of R n. Every such point in R n is called a solution. The collection of all the solution is called the solution set of the equation. By a system of linear equations we mean a finite set of linear equations in finitely many indeterminates. For instance, the following is a system of two linear equations: 2x+3y +4z = 5 x+y +z = 2. (2.4) By a solution of this system we mean a solution of the first equation which is also a solution of the second equation. If A i (i =,2) is the set of solutions of the i-th equation, then the set of solutions of the system is A A 2. In this example, we know that for each i, A i is geometrically a plane. Thus the solution of system (2.4) is the intersection of two planes. We had learnt in school that two planes in R 3 intersect

2 2 2. SYSTEM OF LINEAR EQUATIONS in a line unless they are parallel or conincident. Further, we learn certain adhoc methods to solve a linear system of equations (applicable to a restricted situations); for instance, Cramer s rule. Suppose F is a field. While reading for the first time you may assume that F = R. The set of m n matrices with coefficients from F is denoted by M m n (F). A system of m linear equations in n unknowns x,x 2,...,x n written as a x +a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2 a m x +a m2 x 2 + +a mn x n = b m (2.5) where a i,j F for ( i m, j n), called coefficients and b,b 2,...,b m F, called constant terms of the system. System (2.5) can be described by the following matrix equation: AX = B (2.6) where a a 2 a n b x A = a 2 a 22 a 2n M m n(f), B = b 2 M m (F), and X = x 2... a m a m2 a mn b m x n is the matrix (or column) of unknowns. A is called the coefficient matrix and B the matrix (or column) of constants. The matrix (A B) M m (n+) (F) obtained by attaching the column B with A, is called the augmented matrix of the system. By a solution of AX = B where A and B are as above, we mean an element Y M n (F) such that AY = B. The collection of all the solutions is called the solution set of AX = B. Thus, the solution set of AX = B is a subset of M n (F). Observe that every element of M n (F) defines an element of F n uniquely and vice versa. Thus, often we express a solution of AX = B as an n-tuple rather than an n-column. 2. Three Types of Solutions and Examples Before we talk more about a general system of linear equations, we look at the following three systems with real coefficients. a) x +x 2 +x 3 = 3, x +2x 2 +3x 3 = 6, x 2 +2x 3 = ; b) x +x 2 +x 3 = 3, x +2x 2 +3x 3 = 6, x +x 2 +2x 3 = 4; c) x +x 2 +x 3 = 3, x +2x 2 +3x 3 = 6, x 2 +2x 3 = 3.

3 2. THREE TYPES OF SOLUTIONS AND EXAMPLES 3 We seek solution of these systems in R 3. It is not difficult to verify that system a) has no solution (subtracting the second equation from the sum of the first and the third equation, we get 0 = 2); system b) has only one solution, namely, (,,) (Cramme s rule applicable) and system c) has infinitely many solutions. Indeed, for any real number λ, the tuple (λ,3 2λ,λ) is a solution of system c). Since Crammer s rule is applicable when the number of unknowns and the number of equations are same, and the determinant of the coefficient matrix is nonzero, we require a method to deal with a system of linear equations. In fact, in general, we have three types of solution set to a system of linear equations; namely, an empty set, a singleton set or an infinite set (if the field is infinite). Here wewrite matrices A, X and B for each system of linear equations above. In a), A = 2 3, B = 3 6, X = x x 2 so that 0 2 x 3 (A B) = In b), A = 2 3, B = 3 6, X = x x 2 so that 2 4 x 3 (A B) = In c), A = 2 3, B = 3 6, X = x x 2 so that x 3 (A B) = If the coefficient matrix A belongs to M m n (F), then the column of unknowns has n components so that a solution of the system is an n-tuple which is geometrically a point in F n. In this case, column B of constants must have m components. If B is the zero column, then the system (namely, AX = 0) of linear equations is called a homogenous system. A system which is not homogenous is called a non-homogenous system. A homogeneous system always has a solution (namely, the zero column, i.e., x i = 0 for each i). When scalars are complex numbers geometric description of the solution set is not what we have when scalars are reals.

4 4 2. SYSTEM OF LINEAR EQUATIONS Warning: Before we write the matrix equation we should arrange the unknowns in the same order in every equation of the system. If an unkown is missing from an equation the corresponding coefficient is assumed to be zero. 3. Preliminaries related to Matrices Recall that the collection R of all the real numbers is not just a set. There are two binary operations (namely, addition and multiplications) along with certain nice properties (namely, commutativity, associativity, distributivity of multiplication over addition, difference of numbers and division by nonzero real numbers.) If you are used to the concept of a field then you may assume that F is an arbitrary field. Otherwise, FcanbeassumedtobeRorC(complexnumbers)orQ(rationalnumbers). However, mostly we will use R. We will not deal with the question of finding integer solutions of a system of linear equations. During the process of finding solution of a system of linear equations, we require to perform difference operation, division by nonzero number other than addition; so we always assume that the coefficients are from a field (you may assume F = R or C). To have a geometric idea we consider R as a straight line and R 2 as plane and R 3 as the (3-dimensional) space. If x = (x,x 2,...,x n ),ȳ = (y,y 2,...,y n ) R n and a R, define x+ȳ = (x +y,x 2 +y 2,...,x n +y n ) and a x = (ax,ax 2,...,ax n ). Recall that M m n (F) is the collection of all matrices having m rows and n columns with coefficients from F. Matrices A,B of same size can be added component-wise and the sum is denoted by A+B. Matrix A can be premultiplied to B if number of columns of A is equal to the number of rows of B; in fact, the coefficient in the i-th row and the j-th column of AB is given by n a ik b kj k= if (i,k)-th coefficient of A M m n (F) is a ik and (k,j)-th coefficient of B M n p is b kj, and AB M m p (F). Recall that addition of matrices is commutative, but multiplication is not. A matrix having equal number of rows and column is called a square matrix. Suppose A M m m is a square matrix. Then tr(a) is denotes the sum of the coefficient in the pricipal diagonal (top-left corner to the bottom-right corner); symbolically, tr(a) = m a i,i. (2.7) Definition of det(a) is by induction. If m = (i.e., A is an matrix), then det(a) = A. Induction hypothesis is that we know the definition for square matrices i=

5 4. ROW REDUCED ECHELON (RRE) MATRIX 5 of sizer n n for n m. Define for any j ( j m), det(a) = n a ij b ij (2.8) i= where b ij = ( ) i+j det(a i,j ) and A i,j is the matrix obtained from A by simply removing the i-th row and j-th column. Quantity det(a) is independent of choice of i in the definition. Since we assume readers to have studied the concept of determinant, we will not justify this statement. Another fact is that in the definition of determinant, role of i and j can be interchanged. For instance, a b det = ad bc c d det a, a,2 a,3 a2, a 2,2 a 2,3 a 3, a 3,2 a 3,3 ) a2,2 a = a, det( 2,3 a a 3,2 a,2 det 3,3 a2, a a,3 det 2,2 a 3, a 3,2 a2, a 2,3 + a 3, a 3,3 = a, (a 2,2 a 3,3 a 2,3 a 3,2 ) a,2 (a 2, a 3,3 a,3 a 3,2 )+ a,3 (a 2, a 3,2 a 2,2 a 3, ) Reader can try to verify the following identities: () tr(a+b) = tr(a)+trb; (2) tr(ab) = tr(ba); (3) det(ab) = deta detb. It is hard to verify the last identity. 4. Row Reduced Echelon (RRE) Matrix A matrix is called row reduced echelon (in short RRE) if the following properties hold: () Every zero row is below every nonzero row. (2) The leading coefficient of every nonzero row is. (3) A column which contains leading nonzero entry (which is ) of a row has all other coefficients equal to zero. (4) Suppose the matrix has r nonzero rows (and remaining m r rows are zero). If leading nonzero entry of i-th row ( i r) occurs in the k i -th column, then k < k 2 < < k r.

6 6 2. SYSTEM OF LINEAR EQUATIONS Consider the following matrices: a) is not RRE since it has a zero row 0 0 (namely, 2-nd row) preceding a nonzero row (namely, 3-rd row), b) is notrre since the second row is nonzero but its leading non zero entry is 2 ( ), c) 2 0 is not RRE since the leading nonzero coefficient of the second row is in the secondcolumn but the second column has another nonzero coefficien (in the first row), d) is not RRE since the leading nonzero coefficient of the third 0 0 row is in the third column and the third column has another nonzero coefficients, e) is RRE, f) is not RRE since in this there are two nonzero 0 0 rows (so that r = 2) and k = 2 and k 2 = violating k < k Preliminaries related to Equivalence Relation Goes to apprendix If a reader is aware of the concept of an equivalence relation on a set, then they can either skip this section or continue to read to revisit the same. Let A be a nonempty set. A subset R of A A (the two fold cartesian product of A) is called a relation on A. If (x,y) R, then we write xry and read it as x is related to y. For instance, < is a relation on R. We write x < y for x,y R if x is less than y. Define relation = n in Z: x = n y if n divides x y. () A relation R on A is called reflexive if xrx for every x A. E.g., Equality on C, on R, on R, = n on Z. (2) A relation R on A is called symmetric if xry yrx for any x,y A. E.g. Equality on C, on C, = n on Z. (3) A relation R is called transitive if xry,yrz = xrz for any x,y,z A. E.g., < on R, > on R, Equality on C, = n on Z. A relation on A which is reflexive, symmetric and transitive is called an equivalence relation on A. We are going to describe an equivalence relation (namely, equivalence) on M n (F), n n matrices with coefficients from F.

7 6. ELEMENTARY ROW OPERATIONS 7 6. Elementary row operations An elementary row operation is a map from M m n (F) to itself which is any one of the following three types: () Multiplying the i-th row by a nonzero scalar λ denoted by R i λr i. (2) Interchanging the i-th row and the j-th row denoted by R i R j. (3) For i j, replacing the i-th row by the sum of the i-th row and µ multiple of the j-th row denoted by R i R i +µr j. A row operation is a map from M m n (F) to itself which is a composition of finitely many elementary row operations. a b Example. Let A =. Then, for λ 0 in F, R c d λr produces λa λb c d, R c d R 2 produces and, for µ F, R a b R + µr 2 produces a+λc b+λd. c d Every elementary row operation is invertible. Indeed, () inverse of R i λr i is R i /λr i ; (2) inverse of R i R j is R i R j (self inverse, i.e., inverse of itself). (3) inverse of R i R i +µr j is R i R i µr j. Let ρ be an elementary row operation. Then we have det(a) 0 det(ρ(a)) 0. Indeed, if ρ is R i λr i, then det(ρ(a)) = λdet(a); if ρ is R i R j (i j), then det(ρ(a)) = deta; if ρ is R i R i +µr j (i j), then det(ρ(a)) = det(a). Proposition 6.. Denote an elementary row operation by ρ. If A M m n (R) then where I is the m m identity matrix. ρ(a) = ρ(i) A Thus, if you know the effect of ρ on the identity matrix, then you know its effect on A (namely, by simply premultiplying A by ρ(i)). Readers are suggested to verify this fact in each case. AsquarematrixE ofsizeniscalledanelementary matrixifthereisanelementary row operation ρ such that E = ρ(i n ), where I n is the identity matrix of size n. UsetheaboverulesuccessivelyonamatrixAandgetthefollowingtwostatements for finitely many elementary row operations ρ,ρ 2,...,ρ s (by induction): () (ρ s ρ 2 ρ )(A) = ρ s (I) ρ 2 (I)ρ (I)A. (2) (ρ s ρ 2 ρ )(A) = (ρ s ρ 2 ρ )(I)A.

8 8 2. SYSTEM OF LINEAR EQUATIONS 7. Row Equivalence and Row Equivalent Matrices A matrix A is said to be row equivalent to the matrix B if there are finitely many elementary row operations ρ,ρ 2,...,ρ s such that B = (ρ s ρ 2 ρ )(A). Proposition 7.. Row equivalence is an equivalence relation. () A is row equivalent to itself. Recall that the identity map is composition of two elementary row operations (for instance, composition of R i R j with itself) or R R. (2) A is row equivalent to B if and only if B is row equivalent to A. If B = (ρ s ρ 2 ρ )(A), then A = (ρ ρ s ρ s )(B) and inverse of an elementary row operation is an elementary row operation. (3) If A is row equivalent to B and B is row equivalent to C then A is row equivalenttoc. IfB = (ρ s ρ 2 ρ )(A)andC = (ρ s+t ρ s+2 ρ s+ )(B) then C = (ρ s+t ρ 2 ρ )(A). If A is row equivalent to B we will often write A B. Theorem 7.2. Every matrix is row equivalent to a unique row reduced echelon matrix. Corollary Every equivalence class of row equivalent matrices contains a unique row reduced echelon matrix. Proof. One write an algorithm to find the RRE row equivalent to the given matrix: Step : Appy interchange of rows to push down the zero rows to the end of the matrix so that no zero row is before a nonzero row. Step 2: Find the first nonzero column (from left) (suppose it is k ). Step 3: Again apply interchange of rows to push up a row whose leading nonzero coefficient occurs in the first nonzero column (i.e. in the k -th column), to the first row. Divide the first row by the leading nonzero coefficient so that the leading nonzero coefficient becomes. Step 4. Next apply apply R i R i µr for suitable values of i and µ so that the first nonzero column has nonzero coefficient only in the first row. Step5: Findthefirstnonzerocolumn(fromleft)whenweignorethefirstrow(suppose itisk 2 ). Applyinterchangeofrowstopushuparowwhoseleadingnonzerocoefficient occurs in column k 2, to the second row. Divide the second the second row by the leading nonzero coefficient. Step 6: Apply R i µr 2 for suitable values of i and µ so that k 2 column has nonzero coefficients only in the second row. Step 7: Find the first nonzero row when we ignore the first two rows. continue. When there is no nonzero rows left, stop. Reader can try to show that two distince RRE matrices cannot be row equivalent. This will prove uniqueness part of the statment.

9 8. APPLICATION-I OF ROW REDUCTION: INVERSE OF A MATRIX 9 The RRE matrix row equivalent to A is called row reduced echelon form of A Example 2. We find the RRE form of Apply R 3 R 4. Get Apply R R 2. Get /3 Next apply R /3R. Get Apply R 3 R 3 4R, get 0 0 /3 0 0 / /3. Appy R 2 /4R 2, get 0 0 / /3. Apply R 3 R R 2, get 0 0 / /6. Apply, R 3 6/R 3, get 0 0 / Apply R R R 3 and R 2 R 2 /4R 3 get , which is RRE. Remark. There are more than one ways of getting the row reduced echelon form of a matrix. 8. Application-I of Row Reduction: Inverse of a Matrix Algorithm to find the row reduced echelon form of a matrix, also determines whether a square matrix is invertible and if it is invertible, finds its inverse. Theorem 8.. () Suppose R is row reduced echelon and row equivalent to A. Then A is invertible if and only if R is invertible. (2) An n n RRE matrix is invertible if and only if it is the identity matrix I n. (3) Suppose (A I) is row equivalent to (R B) and R is row reduced echelon. Then A is invertible if and only if R = I and in this case B = A. Proof. Proofs of the first two statements are left to the reader. Suppose ρ is a composition of finitely many elementary row operations such that ρ(a) = R. Then

10 20 2. SYSTEM OF LINEAR EQUATIONS ρ(i) = B. Since R is an RRE, R and hence A is invertible if and only if R = I. Thus ρ(i) A = I so that A = ρ(i) = B. Example 3. Suppose A = 2. Apply elementary row operations on the 2 3 matrix (A I) to find (R B) where R is the RRE matrix row equivalent to A. Apply R 2 R 2 R and then R 3 R on and get Then apply R R R 2 and R 3 R 3 R and get Then apply R 3 /3R 3 and then R R R 3 and /2 /2 get Conclude that that A is invertible and A = /2 /2 2 /2 / /2 /2 9. Application-II of Row Reduction: Rank of a Matrix Suppose A is an n n matrix with coefficients from F. A matrix obtained from A by deleting certain numbers of rows and columns is called a submatrix of A. An integer r 0 but r n is called the rank of A if there is an invertible submatrix of A of size r r but there is no invertible submatrix of A of size (r +) (r +). If A is invertible A has full rank n. The rank of A is zero if and only if A = 0. The rank of a matrix can be found by using the process of row reduction. We record a theorem charcterizing the rank of a matrix whose proof involves only the concept of RRE matrix (or it follows in the section on row rank of Chaper 3). Theorem 9.. () If A and B are row equivalent, then they have the same rank. (2) If A is an RRE matrix, then the rank of A is the number of nonzero rows. (3) The rank of a matrix is the number of nonzero rows of the RRE form of the matrix.

11 0. APPLICATION-III OF ROW REDUCTION: SOLVING SYSTEMS OF EQUATIONS Example 4. The rank of is 3. The Rank of is two. Student must show that this matrix is row equivalent to which is row reduced echelon (so that the rank of the given matrix is 2). Remark 2. The rank of a matrix is a very important parameter. We will see its use in several places in the remainder of the book. 0. Application-III of Row Reduction: Solving Systems of Equations A homogeneous system of linear equations, AX = 0, always has a solution (for instance, x i = 0 i). Suppose we have the following system of linear equations: AX = B. Observe the following statements: Theorem 0.. If (A B) is row equivalent to (A B ), then the two systems AX = B and A X = B have the same solutions. Proof. Observe, ρ(pq) = ρ(i)(pq) = (ρ(i)p)q = ρ(p)q. Thus, we have AX = B if and only ρ(a)x = ρ(b) for any composition ρ of finitely many elementary row operations. Remark 3. If A is a row reduced echelon matrix then one can determine whether AX = B has a solution and when there is a solution, can find all solutions just by inspection. Theorem 0.2. The system of linear equations AX = B has a solution if and only if rank(a) = rank(a B). This system has a unique solution if and only if rank(a) is equal to the number of unknowns. Proof. By Theorem 0., we may assume, A is a row reduced echelon matrix. If the i-th row of A is zero but i-th row of B is nonzero (i.e., b i 0), then AX = B has no solution because the i-th equation of AX = B is 0 = b i which is absurd. Thus we have, rank(a) < rank(a B) = AX = B has no solution. Next suppose rank(a) = rank(a B). Assume that A has r nonzero rows and the leadingnonzerocoefficientofthei-throwisinthek i -thcolumn. Thenbyassumption, b i = 0 for i > r. Let S = {,2,...,n} \{k,k 2,...,k r }. Assign arbitrary values to each x i for i S and find x kj from j-th equation ( j r). Call x k,...x kr dependent unknowns and x i with i S as free unknowns or independent unknowns.

12 22 2. SYSTEM OF LINEAR EQUATIONS If rank(a) = number of unknowns, then there are no free unknowns so that AX = B has a unique solution. Conversely, assume that AX = B has a unique solution. In particular, there are no free unknowns. Thus, n r = 0 = r = n. Now we discuss the three system of equations in Section 2 of the Chapter. Example 5. In a), A = 2 3, B = 3 6, X = x x 2 and (A B) = 0 2 x Applyelementaryrowoperationson(A B)tofindtherowreducedechelonformofthe firstblock. (A B) R 2 R 2 R R R R 2,R 3 R 3 R Since the third row of the first block is zero but the third entry of the second column is nonzero the system has no solution. Example 6. In b), A = 2 3, B = 3 6, X = x x 2 and(a B) = 2 4 x In this case, (A B) R 2 R 2 R,R 3 R 3 R R R R R R +R 3,R 2 R 2 2R 3 Thus, r = 3 = n = m, k i = i for each i =,2,3 so that there is no independent unknown. In fact the solution is x = x 2 = x 3 = (unique).

13 0. APPLICATION-III OF ROW REDUCTION: SOLVING SYSTEMS OF EQUATIONS 23 Example 7. In c), A = 2 3, B = 3 6, X = x x 2 and x 3 (A B) = R 2 R 2 R R R R 2,R 3 R 3 R In this case, n = m = 3, r = 2, k =,k 2 = 2. Thus, x 3 is the independent unknown so assign any arbitrary value to it, say x 3 = λ. Then x λ = 0 and x 2 + 2λ = 3. Thus the general solution of the system is (λ,3 2λ,λ). Thus, there are infinitely many solutions. Example 8. Find λ R and µ R such that the following system of two equations in two unknowns has (i) a solution, and (ii) has a unique solution. x+λy =, 2x+y = µ. λ The augmented matrix of the system is. ApplyingR 2 µ 2 R 2 2R wehave λ. Thus, the system has no solution if 2λ = 0 and µ λ µ 2 In other words, the system is consistent (i.e., there is a solution) when either λ /2 or µ = 2. If λ /2, then there is no free unknown and hence there is a unique solution (for every µ R). Finally, when λ = /2 and µ = 2, the system has an infinitely many solution. Remark 4. If a linear system of equations has more than one solutions then it has infinitely many solutions. (However, this is not valid over a finite field.)