Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value a symbolic expression or another matrix The number of rows and columns in the matrix indicate the size of the matrix Definition : The number of rows in a matrix is called the row dimension of the matrix and the number of columns is called the column dimension matrix with M rows and N columns is called an M N matrix and has MN elements Notice that the row dimension is always stated first followed by the column dimension Throughout this review symbolic matrices will always be denoted using boldface type while the elements of matrices will be put in square brackets For example a 3 4 matrix of ones would be denoted by If a matrix is used in a purely symbolic fashion it may be helpful to indicate its dimensions For example if B is a 4 matrix and C is a 4 4 then the product of B and C might be written as B 4 C 4 4 Note : matrix with only one row and one column is equivalent to a scalar Definition : If we turn every row in a given matrix into a column in a new matrix B the new matrix B is called the transpose of denoted by B T If is M N then B will be N M Example : If 0 then T 3 3 5 4 5 0 4 Definition 3: matrix with M rows and one column is called a vector while a matrix with one row and M columns is the transpose of a vector but is not a vector itself Definition 4: ny ordered array of numbers or expressions can be considered a vector For R L Rankin R /6/98
example the array ( x 3 y z) can be written as the vector v x 3 y z and v T x 3 y z Note : Vectors by default are always matrices with one column Matrices with only one row are the transpose of vectors but are usually not referred to as vectors By convention we will always try to use lower case letters for vectors and their transposes and upper case letters for matrices whose row and column dimensions are both greater than one Notice that if a is an M vector and b is an M vector with the same elements as a they are not equal to one another since they have different row and column dimensions subscript notation is used to refer to the elements of a matrix For example if is a 5 3 matrix the element in row 4 and column would be indicated by 4 gain notice that the row number is the first subscript the column number is the second Example : For the vector v in the last example we have however the element v 4 does not exist v 4 y ( v T ) 4 Matrix ddition and Subtraction Only matrices with equal numbers of rows and columns can be added or subtracted The operation of addition or subtraction is carried out by adding or subtracting the corresponding elements of each matrix Definition 5: If M N has elements ij and B M N has elements B ij then their sum or difference is given by C M N M N ± B M N and the elements C ij of C M N are computed using M N + B M N M N B M N () ij + B ij C ij ij ± B ij i M j N () ij B ij Matrix addition and subtraction have the additional property that they are associative ie they satisfy R L Rankin R /6/98
C + B B + and C B B + (3) Example 3: Find the sum and difference of and B if Solution: The sum is 3 4 a z and B z 4 5 6 while the difference is C B + z + 5 8 a + 5 z + 6 3 3 Multiplication matrix can be multiplied by either a scalar or another matrix and each of these operations are discussed below 3 Multiplying a Matrix by a Scalar When a matrix is multiplied by a scalar the result is a new matrix of the same size as the original where each element of the new matrix is multiplied by the scalar where C B Definition 6: If x is a scalar and is an M N by z + 3 0 a 5 z 6 3 B M N x M N M N x matrix then the product of x with is defined (4) B ij x ij ij x i M j N (5) Example 4: If x 4 and 3 then 4 5 6 B x 6 3 3 4 5 6 4 5 6 6 6 8 4 30 36 3 Multiplying Two Matrices Multiplying two matrices is a bit more complicated than multiplying a matrix by a scalar The basic rules for this type of multiplication are outlined in the definition below Definition 7: Two matrices M N and B P Q can be multiplied if and only if the column dimension of the first equal to the row dimension of the second (ie N P ) The result is a new R L Rankin R 3 /6/98
matrix C R S whose row dimension is equal to the row dimension of (ie R M ) and whose column dimension is equal to the column dimension of B (ie S Q ) Thus Notice that in general matrix multiplication is not commutative ie except in the special case when M P The elements of C are computed using C ij M N B N P C M P C M P M N B N P B N P M N (6) (7) ik B kj i M j P (8) k n equivalent way of thinking of Equation (8) is by taking the elements of row i of and multiplying them by column j of B and adding the results Hence if row i of is and column j of B is b j then Cij ai N a i T b j (9) Example 5: Determine the product of the matrices and B if and B Solution: The product is the 3 matrix C B and since and B satisfy the rules for matrix multiplication then the elements of C are given by Hence and C ij 3 4 5 6 3 ik B kj i j 3 k C C C 3 C C C 3 B + B + 3 B 3 B + B + 3 B 3 B 3 + B 3 + 3 B 33 B + B + 3 B 3 B + B + 3 B 3 B 3 + B 3 + 3 B 33 ( 9) + ( 6) + 3( 3) 30 ( 8) + ( 5) + 3( ) 4 ( 7) + ( 4) + 3( ) 8 4( 9) + 5( 6) + 6( 3) 84 4( 8) + 5( 5) + 6( ) 69 4( 7) + 5( 4) + 6( ) 54 C 30 4 8 84 69 54 9 8 7 6 5 4 3 R L Rankin R 4 /6/98
Note 3: lthough the expressions above are correct it is much easier to think of matrix multiplication in terms of products of the rows of the first matrix times the columns of the second matrix Example 6: Since each row of a matrix is itself a matrix and each column of a matrix is also matrix then we can write any matrix using these concepts For example the matrix B from the previous example can be written in the following ways: where are row matrices of B and are column matrices of B B 9 8 7 6 5 4 r r 3 r 3 and r 9 8 7 r 6 5 4 r 3 3 and c 6 c 5 c 3 4 9 8 7 3 Example 7: If we write the matrices of Example(5) in the forms 3 r B 4 5 6 r then the product of times B can be written in the form 9 8 7 and 6 5 4 3 c c c 3 where B r r c r c r c c c c 3 3 r r c r c r 3 c 3 9 C r c 3 6 30 30 C r c 3 3 8 5 7 C 3 r c 3 3 4 8 8 C r c 4 5 6 9 6 3 4 4 84 84 Hence 8 C r c 4 5 6 5 69 69 C 3 r c 3 4 5 6 7 4 54 54 C r B r c r c r c 3 c c c 3 30 4 8 r r c r c r 3 c 3 84 69 54 R L Rankin R 5 /6/98
The two kinds of multiplication discussed so far are the only kinds defined in classical matrix analysis However we should make note of the fact that some classical vector operations can be represented in matrix notation Definition 8: The scalar (or dot or inner) product between two vectors is defined by a a i + a i + + a N i N and b b i + b i + + b N i N a b a b + a b + + a N b N b a in classical vector notation In matrix notation we have and we see that T T a N a a a N and b N b b b N b b at N b N a a a N a b + a b + + a N b N b N T a N hence b N a b a T b b a b T a (0) Definition 9: The magnitude of a vector is defined by a a a a T a () Definition 0: The cross-product in classical vector notation can only be carried out between two three dimensional vectors The operation is defined by a b ( a b 3 a 3 b )i + ( a 3 b a b 3 )i + ( a b a b )i 3 ( b a) () in classical vector notation while in matrix notation we have where a b b Ba ( b a) T b B T a (3) and 0 a 3 a b a 3 0 a T b b (4) a a 0 b 3 R L Rankin R 6 /6/98
B 0 b 3 b a b 3 0 b B T a a (5) b b 0 a 3 Definition : The outter product between two vectors is defined by If a a i + a i + + a N i N and b b i + b i + + b N i N a b ab T (6) T T a N a a a N and b N b b b N then the outer product of two vectors produces the matrix a a b a b a b N a N bt N a a b b b b a b a b N N C N N (7) a N a N b a N b a N b N 4 Transposes and the Identity Matrix Some basic rules for the transpose operation are listed below In this list and B are real matrices and x is a real scalar ( ) T (8) T ( ± B) T T ± B T ± B T + T (9) ( x) T x T T x (0) ( B ) T B T T T B T () The identity matrix is a special square matrix usually denoted by I which has the following properties: I M M M N M N M N I N N M N () (3) Definition : The elements of the N N identity matrix I are given by i j I ij δ ij i j N (4) 0 i j R L Rankin R 7 /6/98
where δ ij is called the Kronecker delta 5 Some Special Matrices 5 The Zero Matrix The symbol 0 is used for the M N matrix whose elements are all zero and this matrix is sometimes called the null matrix For example the matrix equation B µc D can be written in the alternative form B µc + D 0 where 0 is an appropriately sized zero matrix 5 Diagonal Matrices square matrix is diagonal if non zero elements occur only on the main diagonal Definition 3: The N N matrix with elements ij i j N is diagonal iff ij 0 i j (5) Definition 4: The main diagonal of the N N matrix are the elements and is often displayed as a row matrix in the form The first sub diagonal of are the elements the second sub diagonal of are the elements and the sub diagonal of are the elements The first super diagonal of are the elements the second super diagonal of are the elements and the super diagonal of are the elements D ii i N (6) i i i i i i j th i i j i i i + i i i + i j th i i j + i NN (7) N (8) 3 N (9) j + N (30) N (3) N (3) N j (33) R L Rankin R 8 /6/98
53 Triangular Matrices matrix is lower triangular if all elements above the main diagonal are zero and is upper triangular if all elements below the main diagonal are zero Definition 5: The M N matrix with elements ij is lower triangular if and is upper triangular if ij 0 i < j (34) ij 0 i > j (35) Definition 6: The N N matrix with elements ij i j N is unit lower triangular if ij 0 i < j and ii i N (36) and is unit upper triangular if ij 0 i > j and ii i N (37) 54 Banded Matrices Banded matrices are zero outside a certain band on either side of the main diagonal If the number of nonzero sub diagonals is and the number of nonzero super diagonals is then the bandwidth of the matrix is h L The number of nonzero sub diagonals is number of nonzero super diagonals is h L h U h (38) is called the lower bandwidth of the matrix and the is called the upper bandwidth of the matrix If then h is called the half bandwidth of the matrix Banded matrices are often referred to using the notation ( h L h U ) b h L + h U + h L h U h U Example 8: The matrix given by has a lower bandwidth of h L and an upper bandwidth of b U Hence this is an ( ) band matrix with a bandwidth of b + + 4 0 0 0 0 4 4 0 0 0 6 5 3 0 0 0 6 5 3 0 0 0 6 5 3 0 0 0 6 4 n N N matrix with an ( h L h U ) band can be stored more compact as a matrix with R L Rankin R 9 /6/98
dimensions ( h L + h U + ) N b N Hence instead of requiring N storage locations it can be stored in bn locations For example if is a 00 00 matrix with lower and upper bandwidths of 5 and 7 then instead of requiring 00 0000 storage locations can be stored in ( 5 + 7 + )00 300 storage locations Special techniques exist for solving banded matrix systems Definition 7: tridiagonal matrix is a () band matrix with the property that ij 0 if i j (39) Definition 8: pentadiagonal matrix is a () band matrix with the property that ij 0 if i j 3 (40) 6 Inverse of a Matrix In classical matrix analysis there is no such operation as division between matrices In algebra if we have the scalar equation ax b then we can easily solve this to get b x -- a a b ba where a a However if we have the matrix equation x b then x b has no meaning in classical matrix analysis In fact the solution to x b is usually written as x b b where ( ) is called the inverse of Definition 9: If is an N N matrix then the inverse of is the N N matrix B which satisfies (4) where I is the N N identity matrix Symbolically B hence we normally write Some properties of the inverse are shown below B B I I (4) ( ) ( T ) ( x) (43) ( ) T (44) x x ( B ) ( + B) B + B -------- (45) (46) (47) R L Rankin R 0 /6/98
Note 4: Not all square matrices have an inverse Definition 0: If the N N matrix does not have an inverse is said to be singular If does have an inverse the matrix is said to be non singular In the following sections we will see the conditions necessary for an inverse to exist 7 Determinant of a Matrix The determinant is an operation on a square matrix which produces a scalar Later on we will see that the determinant is related to the question of invertibility Definition : If is a matrix whose only element is then the determinant of is defined by If is a matrix given by then the determinant of is defined by If is a 3 3 matrix given by det( ) (48) det( ) (49) 3 3 (50) then the determinant of is defined by 3 3 33 det( ) 33 + 3 3 + 3 3 ( ) ( 3 3 + 33 + 3 3 ) (5) For matrices beyond below 3 3 the determinant is defined in terms of other operations as shown Definition : If is an N N matrix with elements ij then the minor of ij is the determinant of the ( N ) ( N ) matrix formed by deleting the i th row and j th column of and is denoted by Every element of has a minor M ij Definition 3: If is an N N matrix with minors M ij then the cofactor of ij is the number R L Rankin R /6/98
defined by C ij ( ) i j + M ij (5) The cofactors themselves form an N N matrix C called the cofactor matrix of Definition 4: If is an N N matrix with elements ij and cofactors C ij then the determinant of is defined by det( ) N ij C ij expansion using row i j N ij C ij expansion using column j i (53) Example 9: Compute the determinant of the matrix by expanding row 3 Solution: If we expand using row then The minors for row 3 are M 3 det 0 6 M 3 det 4 0 M 33 det 4 4 and the corresponding cofactors are C 3 ( ) 3 + M 6 C 3 3 ( ) 3 + M C 3 33 ( ) 3 + 3 M 33 4 Hence the determinant is 3 4 0 0 3 4 3 det( ) 3 C 3 + 3 C 3 + 33 C 33 0 3 det( ) ( 6) + 4( ) + ( 3)4 7 0 Definition 5: If is an N N matrix with cofactor matrix C then the adjoint of is the N N matrix defined by adj( ) ( C ) T (54) R L Rankin R /6/98
Definition 6: Cramer s rule for determining the inverse of is defined by --------------- adj( ) (55) det( ) lthough Cramer s rule produces an exact inverse in infinite precision arithmetic it is probably the worst method that you can use in finite precision arithmetic (the arithmetic of computers and calculators) However it does show us that if det( ) 0 then the inverse of does not exist Hence we see that If det( ) 0 then is singular If det( ) 0 then is non singular matrices and x is a real scalar the determinant has the following prop- If and B are real N N erties: 8 Condition of a Matrix det( T ) det( ) (56) det( x) x N det( ) (57) det( B ) det( )det( B) (58) det( ± B) det( ) ± det( B) (59) det( ) When solving systems of linear equations of the form --------------- (60) det( ) x b and we know that if det( ) 0 then does not exist and the system either has no solution at all or it has no unique solution lthough the determinant gives the condition for an exact singularity it does not give a good indication of how close a matrix is to being singular For example consider the system x b where 600 800 3000 4000 and b 00 Since det( ) 400 seems to be very large we might conclude that this matrix is not very close to being singular If we use Cramer s rule and exact arithmetic the inverse of is x 000 b the symbolic solution is adj( ) 3000 600 --------------- ------------------------------------- det( ) 400 4000 800 and the corresponding exact solution to this system is 000 --------------- 00 3000 --------------- 400 3 -- 000 --------------- 00 x b 3000 --------------- 400 3 -- 00 000 R L Rankin R 3 /6/98
However if we change b from 00 to 0 we find that the solution changes to x 000 --------------- 00 3000 3 --------------- -- 400 0 000 99005 74005 Definition 7: For the system x b whenever small changes in either or b lead to large changes in x the system is said to be ill conditioned and even though the determinant seems large the matrix is nearly singular Clearly the determinant is not a good measure of how close a matrix is to being singular However there is another measure of how close a matrix is to being singular called the condition number of the matrix Definition 8: The condition number of an N N matrix is defined as where is called the norm of and is defined by (6) (6) If we were able to perform all operations in infinite precision arithmetic the every matrix would always have a condition number of exactly one However since all calculator and computer operations are carried out in finite precision arithmetic some matrices will have conditions number much larger than one If the condition number of a matrix is small then the matrix is well conditioned and if it is large then the matrix is ill conditioned We can use the following rule of thumb: if cond( ) 000 then the matrix is well conditioned (far from being singular) if cond( ) 0000 then the matrix is ill conditioned (close to being singular) 3 if 000 < cond( ) < 0000 then the matrix is in the gray area and may be either s with all rules of thumb it is NOT always valid In practice we do not use the formal definition of the condition number to compute its value since this requires computing the inverse There are special techniques for estimating the condition of a matrix 9 Trace of a Matrix cond( ) max j i The trace is another operation on a square matrix which produces a scalar N ij Definition 9: If is an N N matrix then the trace of is simply the sum of all the diagonal R L Rankin R 4 /6/98
elements of and is defined by N tr( ) ii i (63) 0 Partitioned Matrices Earlier we discussed matrix multiplication by thinking of the rows of the first matrix as row matrices and the columns of the second matrix as column matrices This process was a form of matrix partitioning Whenever we break a matrix up into submatrices this is referred to as partitioning s an example consider the matrix This matrix can be partitioned in the from 3 4 5 6 7 8 9 0 where 3 4 5 6 7 8 9 0 a a 3 T 4 a 4 at 3 9 0 and 4 Partitioning is advantageous when we want to isolate parts of a matrix to emphasize the roles those parts play in matrix equations For example it can be shown that if the 6 6 matrix is partitioned in the form and we define and 3 5 6 7 then the inverse of is given by 8 P 4 4 Q 4 6 6 R 4 S W S R 4 P 4 4 Q 4 X 4 4 P 4 4 + P 4 4 Q 4 W R 4 P 4 4 X 4 4 P 4 4 Q 4 W 6 6 W R 4 P 4 4 W This means that we can invert a 6 6 matrix by inverting a 4 4 matrix P and a matrix W R L Rankin R 5 /6/98
Basic Theory Part B Systems of lgebraic Equations We now consider systems of linear algebraic equations For example if we have a system with M equations in N unknowns it can always be written in the form x + x + + N x N b x + x + + N x N b M x + M x + + Mn x n b M () In order to simplify the task of dealing with such systems we use matrix notation and define the M N coefficient matrix N N M M MN () the M right-hand-side (rhs) vector and the N vector of unknowns hence the system can be written in the compact form and x (3) x b (4) This represents 3 fundamentally different kinds of systems: If M < N then we have fewer equations than there are unknowns and the system is said to be under-determined In this case either no solution exists or an infinity of solutions exist If M > N then we have more equations than there are unknowns and the system is said to be over-determined In this case either no solution exists or an infinity of solutions exist 3 If M N then we have equal numbers of unknowns and equations and the system is said to be determined and is represented by a square matrix In this case either no solution exists or a unique solution exists In this chapter the only case we consider is the case where M N and therefore if det( ) 0 then exists and the unique solution is given by b b b b M x (5) If det( ) 0 then does not exist and there is no solution to the system Computing the inverse of a matrix is computationally intensive and is often a very inefficient way to solve a system of equations In the next sections we consider alternative methods of solving systems without ever computing the inverse of the matrix b x x x N R L Rankin R 6 /6/98
Gaussian Elimination Gauss elimination is carried out by performing elementary row operations (EROs) on the matrix and its right hand side There are three elementary row operations and the basic definitions relating to them are given below Definition : For an M N matrix the three elementary row operations are: ERO : interchange any two rows of ERO : multiply any row of by a nonzero scalar c ERO 3: replace the i th row of by the sum of the i th row of plus c times the j th row of for i j Theorem : Performing any sequence of EROs on the system solution x x b does not change the In order to perform EROs we define the following elementary vectors and matrices Definition : n N elementary vector e is a column vector of zeros except in the i th i row where there is a one Example : If N 6 construct e 4 Solution: This vector is simply the 6 e 4 zero vector with a in the 4th row hence 0 0 0 0 0 T Definition 3: If is an M N matrix then interchanging the i th and j th rows of (ERO ) is equivalent to E where is the matrix obtained by interchanging the i th and j th ij E ij M M rows E ij of the M M identity matrix I The elementary matrix E ij has the property that E ij Definition 4: If is an M N matrix then multiplying the i th row of by a nonzero constant c (ERO ) is equivalent to Q i ( c) where Q i ( c) is the M M matrix The elementary matrix Q i ( c) Q i ( c) I + ( c )e i et i (6) has the property that [ Q i ( c) ] Q -- (7) i c R L Rankin R 7 /6/98
Definition 5: If is an M N matrix then adding c times the j th row of to the i th row of (ERO 3) is equivalent to R ij ( c) where R ij ( c) is the M M matrix The elementary matrix R ij ( c) R ij ( c) I + ce i et j i j (8) has the property that Summary: We have the following equivalences [ R ij ( c) ] R ij ( c) (9) E is equivalent to interchanging the i th and j th ij rows of Q is equivalent to multiplying the i th i ( c) row of by c 0 3 R is equivalent to adding times the j th row of to the i th ij ( c) c 0 row of Quick Tips: If is an M N matrix we can quickly construct Q by replacing the i th i ( c) diagonal element of the M M identity matrix I by c Similarly we can quickly construct R ij ( c) by replacing the ij th element ( i j) of the M M identity matrix I by c Example : Replace row of the matrix by row minus 45 times row 4 3 4 0 3 5 0 3 Solution: The elementary matrix tip R 4 ( 45) performs this operation and according to the quick Hence R 4 ( 45) 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 which you can verify is the correct result 3 4 0 3 5 0 3 3 4 35 45 55 5 0 3 Example 3: Multiply the 3rd row of the matrix from the last example by 6 Solution: The elementary matrix Q 3 ( 6) performs this operation and according to the quick tip R L Rankin R 8 /6/98
Hence Q 3 ( 6) 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 which you can verify is the correct result 3 4 0 3 5 0 3 3 4 0 3 30 6 0 3 Definition 6: Simple Gaussian Elimination for the system N N x N b N consists of two steps Step called forward elimination consists of performing ERO s on the augmented matrix N ( N + ) N N b until N is upper-triangular Step called backward elimination consists of solving the upper-triangular system obtained in Step for the solution vector starting with the last equation of the upper-triangular system and using simple substitution x N Definition 7: Simple Gauss-Jordan Elimination for the system consists of two steps Step also called forward elimination is identical to Step of simple Gaussian elimination Step consists of performing additional ERO s on the upper-triangular system obtained in Step until the augmented matrix is the N N identity matrix in the first N columns N ( N + ) I N N x N and is equivalent to the backward elimination of Gaussian elimination N N x N b N (0) Definition 8: The inverse of the matrix N N can be obtained by a slightly modified version of Gauss-Jordan elimination The process consists of performing ERO s on the augmented matrix N N N N e e e N until the first N columns of is the N N identity matrix t this point the last N columns of () will be the N N inverse of N N N N I N N N N () Example 4: Solve the system R L Rankin R 9 /6/98
using elementary matrices and (a) Gaussian elimination (b) Complete the operations necessary for Gauss Jordan elimination (c) Using the results of part (b) to invert the coefficient matrix for this system Solution: (a) The augmented matrix is x + y z 4x x y + z 5 y + z and all the operations are shown below: 0 4 5 R 0 0 -------- 0 R ( ) 0 0 4 5 0 0 0 3 6 3 R 3 0 0 3 -------- R 3 ( ) 0 3 6 3 0 0 0 0 3 6 3 0 3 3 R 3 3 -------- R 0 0 -- 3 3 0 0 0 3 6 3 0 3 -- 0 3 which completes the forward elimination The back substitution then gives the gauss elimination solution x (b) If we want to perform Gauss Jordan reduction we continue with 0 3 6 3 0 0 4 R 3 0 0 3 -------- 3 R 3 ( 6) 3 0 3 6 3 33 0 6 0 0 0 0 5 R 3 0 3 -------- 4 R 3 ( ) 4 0 3 0 3 33 0 0 0 0 0 0 0 3 0 3 0 0 0 3 0 3 0 3 0 0 6 R -------- R -- 5 3 3 -- 0 0 3 5 0 3 0 3 0 0 0 0 0 0 0 0 0 3 0 3 0 0 -- 0 0 7 Q -- 0 0 0 0 Q -- 3 Q3 ( ) 6 0 3 -- 0 3 0 3 0 0 0 0 0 0 0 0 0 which completes the solution Notice that we can do more than one operation at a time as in I x R L Rankin R 0 /6/98
For Gaussian elimination the entire sequence of operations for the forward elimination can be summarized as and for Gauss Jordan reduction the sequence is where 0 0 -- 0 0 0 0 0 3 -- 0 0 0 0 0 0 0 0 0 0 0 Q -- Q -- 3 Q3 ( ) 6 3 0 0 3 R 3 -- 3 R3 ( )R ( ) 0 0 3 -- 0 0 0 3 -- 0 0 0 7 Q -- Q -- 3 Q3 ( )R -- 3 R3 ( )R 3 ( 6) 0 0 R 3 -- 3 R3 ( )R ( ) 0 P 0 0 0 0 0 P Q -- Q -- Q3 ( )R 3 -- 3 R3 ( )R 3 ( 6)R 3 -- R3 ( )R 3 ( ) Notice that -- 0 0 0 3 -- 0 0 0 3 -- 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 3 -- 0 0 0 0 0 0 0 0 0 0 -- 6 -- 0 6 0 -- 3 3 -- P 0 -- -- 0 6 6 0 -- 3 3 -- 4 5 0 0 0 0 0 0 (c) We have also performed all of the operations necessary to invert If we form the augmented matrix 7 I x then B 0 I 0 0 4 0 0 0 0 hence -- -- 0 6 6 PB 0 0 4 0 0 0 0 0 -- 3 3 -- 0 0 0 0 0 0 6 -- -- 0 6 0 0 3 -- 3 -- I R L Rankin R /6/98
which we can check with -- 0 0 6 6 4 6 -- 0 0 6 4 -- 6 0 0 6 6 4 6 0 6 0 0 0 6 I 3 n lgorithm for Naive Gaussian Elimination If we let x b be an N N system of equations where and b are known To solve this system perform the following three steps I Form augmented matrix: b and designate the rows of as r r N each N then II Perform Forward Elimination perform the following elementary row operations on each column of in the order indicated: Step Eliminate column of by performing the N elementary row operations r -------- 3 r r 3 -------- N r r N --------- r Step Eliminate column of by performing the N elementary row operations 3 r 3 -------- 4 r r 4 -------- N r r N --------- r Step 3 Eliminate column 3 of by performing the N 3 elementary row operations r 43 4 -------- r 3 r 53 5 -------- r 3 r N3 N --------- r 3 33 33 Step N - Eliminate column N of by performing the following N ( N ) elementary row operation N N r N ------------------------- r N III Backward Elimination upon completing II the matrix will be upper triangular and r r r N N N 33 R L Rankin R /6/98
starting with the last equation we perform the following operations in the order indicated: The Gauss elimination process described above fails immediately if any main diagonal element is zero or becomes zero during the forward elimination steps In some cases this may mean that no solution to the problem exists in others it is simply an artifact of the solution pro- Step x N ------------------- N N + Step N + N N x ------------------------------------------------------------ N Step 3 N + N N x N N N x --------------------------------------------------------------------------------------------------------- N Step N - x N + 3 x 3 4 x 4 N x N N x N -------------------------------------------------------------------------------------------------------------------------------------------- Step N x N + x 3 x 3 N x N N x N -------------------------------------------------------------------------------------------------------------------------------------------- Forward Elimination of : The following is pseudo-code for forward elimination of without using the augmented matrix k to n - i k + to n L ik ik kk j k to n next j next i next k Forward Elimination of b: Pseudo-code for the forward elimination of b j to n - i j + to n b i b i L ij b j next i next j Backward Elimination: Pseudo-code for the backward elimination of b to get x (in this form of the code x is stored in b) j n downto b j b j jj i to j - next i next j 4 Pivoting NN x N N N N x N N ij ij L ik kj b i b i ij b j N N R L Rankin R 3 /6/98
cess which can be avoided by interchanging rows of the system Definition 9: ny element on the main diagonal of a matrix is called the pivot element and if the row (or column) containing the pivot is interchanged with any other row (or column) the process is called pivoting and pivoting can always be represented by the elementary matrix If both rows and columns are pivoting the process is called full pivoting and if only rows or only columns are pivoted the process is called partial pivoting E ij Definition 0: Partial row pivoting is carried out by first identifying the element below the main diagonal in the pivot row which is largest in absolute value The row that this element is in and the pivot row are then interchanged to accomplish the pivoting Example 5: Solve the following system using gauss elimination with partial row pivoting Solution: First form the augmented matrix 0 4 3 3 x 5 3 5 Since then we must pivot before carrying out the elimination of the first column Since row has the largest element in absolute value below get 0 0 b 0 5 4 3 3 3 5 then we interchange rows and to and then eliminate the rest of column : E 0 4 3 0 5 3 3 5 lthough 0 R 3 ( ) 4 3 0 5 if pivoting is being used it is used every time we eliminate a new column Thus since 7/ (in row 3) is larger than the (in row ) we must interchange rows and 3 to get 0 7 -- 7 -- 7 -- Elimination of column then yields 3 E 3 4 3 0 7 -- 7 -- 7 -- 0 5 R L Rankin R 4 /6/98
and the solution to Ux b 4 4 R 3 ( 4 7) 3 7 7 is 4 3 0 7 -- -- -- 0 0 3 3 x T U b 4 5 Scaling a Matrix If the ratio of the largest element in absolute value to the smallest element in absolute value is a large number (> 00) then significant round-off errors may occur In such cases it may be beneficial to scale the matrix Definition : Scaling is carried out before pivoting by dividing the elements of each row of the augmented matrix by the largest element in that row (excluding ) b i Example 6: Solve the system (a) exactly Using 3 significant figures in all calculations solve the system (b) without scaling and (c) with scaling and compare the results Solution: (a) the exact solution is 3 05 3 03 3 notice that no pivoting is necessary for the first column x 04 98 3 x T For (b) we form the augmented matrix and b 3 05 04 3 03 98 3 3 R 0667R R 3 0333R Since pivoting is not required for the second column either then 3 05 04 0 433 330 86 0 0334 30 36 3 05 04 0 433 330 86 0 0334 30 36 R 3 + 0077R Backward elimination yields the result 3 05 04 0 433 330 86 0 0 95 94 For (c) we first scale the matrix to get 0844 56 x 094 with errors ε 760 % 0997 03 R L Rankin R 5 /6/98
b 3 05 04 3 03 98 3 3 then elimination of columns and gives R 05 R 03 R 3 3 0086 0090 00 099 0094 009 00 095 0333 0333 00 00 0333 0333 00 00 0 00485 094 0894 0 0 077 078 The scaling process significantly improves the results 0990 x 0990 ε 00 00 00 0 % R L Rankin R 6 /6/98