FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 8 MATRICES III Rank of a matrix 2 General systems of linear equations 3 Eigenvalues and eigenvectors Rank of a matrix Sets of n simultaneous linear equations in n unknowns in the form AX b were considered in module 7 When the determinant of A is non-zero ie A it proved easy to obtain solutions However when A it is much harder and to clarify the position it is helpful to introduce the concept of the rank of a matrix Consider the system AX b To solve this system using the elimination method discussed in module 7 you reduce the matrix A to upper triangular form with non-zero elements along the principal diagonal This procedure is possible when A However the rearrangement fails when A since in this case the matrix can only be reduced to echelon form An example of a matrix in echelon form is shown below with the dots denoting any non-zero quantities whereas the letters denote quantities which can be zero or non-zero: a b c d e f Observe that in this echelon form the first non-zero elements in each row do not all lie on the principal diagonal as they do when the matrix is in upper triangular form Def The number of non-zero rows in the echelon form of a matrix A equals the rank of A In the above example the rank of the matrix is 3 It is important to point out that rank is defined for matrices of all orders not just square matrices The rank of a matrix gives the number of essentially different rows in the matrix ie rows which cannot be written as linear combinations of some or all of the other rows Understanding this point is important in understanding the different types of solution that are possible for general sets of equations of the form AX b [The usual mathematical definition of rank is that it is the number of rows in the largest square sub-matrix with non-zero determinant A square sub-matrix is found from A by deleting a number of rows and columns to obtain a square matrix It is possible to show that the two definitions of rank are identical Calculating the rank using sub-matrices can be tricky when the matrix is large and in practice it is usually better to adopt the echelon approach] Forasystem AX b define the augmented matrix A : b as the matrix A with the column vector b added to it as an extra column With this definition we can state the following result Result If rank A rank A : b then AX b has NO solution equations are inconsistent; if rank A rank A : b then AX b has a solution equations are consistent and this solution has s free parameters where s number of unknowns rank A
Ex Find the rank of the matrix A 2 3 2 2 We now use the elimination process to reduce the matrix A to echelon form by considering each column in turn Leaving the first row unaltered we first ensure the remaining non-zero elements in the first column become zero; then leaving the first two rows unaltered we make zero any non-zero elements in the bottom part of the second column continuing this process column by column until the matrix is in echelon form With this procedure A row 3 row row 4 2 row 3 3 3 3 row 5 2 row 2 2 2 2 row 3 row 2 row 4 3 row 2 row 5 2 row 2 The matrix is now in echelon form and the rank is the number of non-zero rows Hence it follows that rank A 2 2 General systems of equations In real situations the systems will contain a large number of equations but to illustrate the ideas let us consider very small systems a Under-specified sets of equations number of equations less than number of unknowns Case i Solve x y 2 3 2 z This set is under-specified because when the matrices are multiplied out we have two equations in three unknowns Forming the augmented matrix and then reducing it to echelon form gives 2 3 2 row 2 row 2 The rank of the augmented matrix the number of non-zero rows in the full reduced matrix above is clearly 2 Ignoring the final column gives the matrix A and the number of non-zero rows in the reduced form of this is again 2 Since the ranks are equal the set of equations is consistent The reduced set of equations can be read off from the reduced form of the augmented matrix the first three columns giving the coefficients of the unknowns on the LHS whereas the final column provides the new column vector on the RHS Thus the reduced system is x + y + z y+2z 2
Put z C a constant then the second equation implies y 2C Substituting these quantities into the first of the reduced equations then yields x y z 2C C C Thus the solution is x y z C 2C C for any constant C [Note that in the above example the number of unknowns in the original set of equations 3 rank A 2 and so the general result stated earlier implies there is one free parameter namely the constant C in the above solution] Case ii Solve x y 2 2 2 z Here again there are two equations in three unknowns Reducing the augmented matrix gives 2 2 2 row 2 2 row In this case the rank of the augmented matrix is 2 but the rank of the matrix A is These values are not the same and so the system does NOT have a solution the set of equations is inconsistent which is obvious of course from the original equations b Over-specified sets of equations more equations than unknowns Case iii Solve 2 x y 3 The system has three equations in two unknowns Reduction of the augmented matrix leads to 2 row 2 - row row 3 - row 3 2 2 row 3 2 row 2 Hence rank A 2 and rank A : b 2 the same and so the equations are consistent The reduced system is x + y y Hence it follows that y and x y 2 and the solution is x y 2 [Number of original unknowns 2 rank A 2 so there are 2 2 free parameters in the above solution] In all cases the rank of the augmented matrix gives the number of independent equations in the given system ie equations which cannot be obtained as combinations of other equations in the system 3
Case iv Solve 2 x y 3 2 Again there are three equations in two unknowns In this case manipulation of the augmented matrix leads to 2 row 2 row row 3 row 3 2 2 2 row 3 2 row 2 4 Thus rank A 2 and rank A : b 3 which are different The equations are inconsistent with no solution For the above four systems of equations the types of solution are easily spotted However the situation is much less obvious if you consider larger systems say equations in 9 unknowns The procedure discussed above provides a framework for determining the appropriate solution for systems of all sizes 3 Eigenvalues and eigenvectors The theory of eigenvalues is of importance in many physical problems eg considering the stability of numerical methods for solving differential equations and determining the normal modes of oscillation of vibrating systems a 2 2 matrix Let us illustrate the method with a 2 2 matrix A Suppose a a A 2 X a 2 a 22 then consider the eigenvalue equation where λ is a scalar a number Equation can be written x AX λx AX λx AX λix A λix 2 where I is the unit 2 2 matrix Written in full 2 is a set of two simultaneous linear equations in two unknowns: a λ x + a 2 3 a 2 x + a 22 λ Obviously the two equations have the trivial solution x ie X but this solution is usually of no physical interest However for particular values of λ non-zero solutions for x and may exist and these often do have physical significance Equations 3 can be written BX whereb A λi andyou know from module 7 that for this set of equations to have a non-zero solution it is necessary that B For the 2 2 matrix A condition 4 becomes a λ a 2 a 2 a 22 λ ie a λa 22 λ a 2 a 2 ie A λi 4 4
λ 2 a + a 22 λ + a a 22 a 2 a 2 5 This is a quadratic equation in λ known as the characteristic equation of A and the left-hand side is called the characteristic polynomial of A The two solutions of equation 5 λ λ 2 say normally distinct are called the eigenvalues of A In some texts the word eigenvalue is replaced by characteristic value or latent root When the solution λ λ is substituted back into the system 3 the resulting equations can be solved to produce non-zero solutions for x and called the eigenvector corresponding to the eigenvalue λ In a similar way you obtain the eigenvector corresponding to the eigenvalue λ 2 Let us now illustrate the above procedure for a particular 2 2 matrix 2 E Find the eigenvalues and eigenvectors of the matrix A 3 We need to solve the matrix equation AX λx or A λix Non-zero solutions for λ exist only if A λi ie λ 2 3 λ λ3 λ 2 λ3 λ yielding λ 3 these are the eigenvalues We now calculate in turn the eigenvector corresponding to each eigenvalue For λ it is necessary to solve A IX 2 x 3 2 2 x which gives the two equations 2 and 2 These are obviously identical and both are satisfied when There is no restriction on x however which can have any value C say Hence the x C eigenvector is C for any constant C For the other eigenvalue λ 3 a similar procedure is adopted The system A 3 IX corresponds to 3 2 x 3 3 2 2 x leading to 2x +2 Here choose D an arbitrary constant then it follows that x D x D Hence the eigenvector is D for any constant D D Two comments should be made about the above solution Firstly eigenvectors always contain at least one free parameter similar to C and D in the above example If your answer does not possess this feature you have made a mistake Secondly you can always check that the answers satisfy the eigenvalue equation AX λx In the above example for instance the eigenvector corresponding to the eigenvalue 3 satisfies AX 2 D 3 D Similarly for the other eigenvector you obtain 2 C 3 D+2D +3D C+ + 5 3D 3D 3 C D D 3XλX C
b n n matrix The theory presented above for a 2 2 matrix can easily be extended to a general n n matrix Let A be an n n matrix and X be an n column matrix defined by a a 2 a n x a A 2 a 22 a 2n x and X 2 a n a n2 a nn x n Then the equation AX λx whereλis a number becomes A λi X whereiis now the unit n n matrix Written out in full we have a set of n simultaneous homogeneous equations in the n variables x x n : a λx + a 2 + + a n x n a 2 x + a 22 λ + + a 2n x n 6 a n x + a n2 + + a nn λx n The above system always has the trivial solution x ie X but non-zero solutions can exist for values of λ which satisfy: a λ a 2 a n a A λi 2 a 22 λ a 2n 7 a n a n2 a nn λ Condition 7 again leads to the characteristic equation of A which is now a polynomial equation of degree n in λ The n roots λ i i 2n of 7 which are not necessarily all distinct are called the eigenvalues of A If a particular eigenvalue λ i is substituted back into 6 the resulting equations can be solved for x x n to give the eigenvector always non-zero corresponding to the eigenvalue λ i Let us work through an example for a 3 3 matrix Ex 3 Find the eigenvalues and eigenvectors of the matrix A 2 3 2 2 Need to solve A λi λ 2 3 ie 2 λ 2 λ We know the characteristic equation here will be cubic since A is a 3 3 matrix and there are two ways to proceed The determinant can be expanded directly but in order to find the eigenvalues it will then be necessary to determine the roots of the third order polynomial This is usually not too difficult but it usually involves spotting factors and can be tricky The alternative approach is to manipulate the elements in the determinant before the evaluation in order to simplify the calculations and pull out common factors involving λ Both methods will be given below Direct way Expanding the determinant by the first row λ { 2 λ λ } 2{ λ 2} +3{ 2 λ2} λ{λ 2 +λ 2+} 2{ λ+2}+3{+4+2λ} λλ 2 +λ 23 λ+32λ+5 ie λ 3 λ 2 + λ 6+2λ+6λ+5 λ 3 +λ 2 9λ 9 6
This cubic must now be factorised so it is usually necessary to spot a value of λ which satisfies the equation say λ a then use the remainder theorem which states that λ a is then a factor of the cubic Here you could spot that λ satisfies the cubic but it is easier to observe that the cubic can be rearranged to display a common factor: λ 3 + λ 2 9λ 9λ 2 λ+ 9λ +λ+λ 2 9 therefore λ +λ 2 9 λ +λ 3λ +3 ie λ 3 3 three eigenvalues Alternative method We spot that adding together the first and third columns of A λi produces a zero element and two others with a common factor dependent on λ so 3 λ 2 3 A λi 2 λ 3 λ λ col + col3 2 3 3 λ 2 λ λ 3 λ 2+λ 2 λ row -row 3 λ Notice that we have carefully recorded the row and column operations we have carried out very useful if you need to check your working subsequently! Only one element now remains in the first column and a common factor has already been extracted hence A λi 3 λ 3+{ 2 + λ 2 λ } 3 λ +2+λ 2 3 λ +4+4λ+λ 2 3 λ3 + 4λ + λ 2 3 λ + λ3 + λ and the same eigenvalues are derived as using the direct method Whichever way you have found the eigenvalues the eigenvectors must be calculated by solving the matrix equation A λix λ 3 Need to solve A 3IX which can be written A +3IX The set of equations to be solved found by adding 3 to the diagonal elements is 3x +2 +3x 3 x + x 3 2x + +4x 3 It is convenient to interchange the first and second equations giving x + x 3 3x +2 +3x 3 2x + +4x 3 Then using eqn2 3 eqn and eqn3 2 eqn the system becomes x + x 3 +6x 3 +6x 3 7
Clearly the second and third equations are identical so the last one can be ignored If we choose x 3 C where C is any non-zero constant then the second equation implies 6Cand after substitution into the first equation x 5C Hence the eigenvector corresponding to λ 3 is x 5C 6C x 3 C λ Solve A IX or A + IX equivalent to x +2 +3x 3 x x 3 2x + +2x 3 Subtract the first equation from the second equation and 2 eqn from the third equation to give x +2 +3x 3 3 4x 3 3 4x 3 Again the last two equations are identical On putting x 3 D we find that 4 3 Dandthenx 3 D so that the eigenvector corresponding to λ is x D/3 4D/3 for any D x 3 D λ 3 In this casewe must solve A 3IX For the two previous eigenvalues we found the eigenvectors by manipulating equations but we could easily work with a matrix representation We shall illustrate the matrix method for this third eigenvalue Thus we need to find the solution of 3 2 3 x 5 2 2 x 3 and interchanging the first and second rows equivalent to interchanging the first and second equations gives 5 x 3 2 3 2 2 x 3 Simplifying using the elimination method by calculating row 2 + 3 row and row 3-2 row leads to 5 3 x x 3 This is equivalent to the reduced system x 5 x 3 3 + 8
The last pair of equations imply and hence using the first equation x x 3 Choosing x 3 E we obtain x E for any E x 3 E It is important to note that there is more than one way in which to write the eigenvectors For instance in finding the eigenvector corresponding to λ 3 in the above Example we could have chosen F in which case it follows that x 3 F/6 and hence x 5F/6 In this case it is easily seen that the new solution is identical to the previous one after a redefinition of the arbitrary constant F 6C Concluding comments Before finishing the module two further points are briefly mentioned i It can be shown that if A is symmetric a situation which occurs in many applications then the eigenvalues are real and the eigenvectors corresponding to different eigenvalues are perpendicular ii In all the cases considered in this module the roots of the characteristic equation are real and distinct Clearly this is not always the case When some or all of the roots are equal then the calculation of the eigenvectors is more complicated This more difficult case is not considered in this module rec/lm3 9