Fundamentals of Engineering Analysis (650163)

Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud

Matrices: Introduction DEFINITION A matrix is any rectangular array of numbers or functions o o o o o o o o o o a a n a m a mn The number of functions are called entries or elements of the matrix m stands for the number of rows n stands for the number of columns The size of the matrix is (m n) An (n n) matrix is called a squared matrix or a matrix of order n. A ( ) matrix is simply a constant or single function. A (m ) matrix is called a column vector. A ( n) matrix is called a row vector. Matrices are denoted by capital boldface letters A, B, C,, or by writing the general entry in brackets; thus A = [a jk ], and so on. Vectors are denoted by lowercase boldface letters a, b, or by its general component in brackets, a = [a j ], /7/8 Dr. Omar R Daoud

Matrices: Introduction Introduction The first matrix in has two rows, which are the horizontal lines of entries. Furthermore, it has three columns, which are the vertical lines of entries. The second and third matrices are square matrices, which means that each has as many rows as columns 3 and, respectively. a a a. 6 a a a 3 3 33 x 4 e x, a a a, 6x 3 e 4x 3.3 5, a a a 3, /7/8 Dr. Omar R Daoud 3

Matrices: Introduction Introduction The entries of the second matrix have two indices, signifying their location within the matrix. The first index is the number of the row, and the second is the number of the column, a a a. 6 a a a 3 3 33 x 4 e x, a a a, 6x 3 e 4x 3.3 5, a a a 3, together the entry s position is uniquely identified. For example, a 3 (read a two three) is in Row and Column 3, etc. The notation is standard and applies to all matrices, including those that are not square. /7/8 Dr. Omar R Daoud 4

Matrices: Introduction Introduction The fourth matrix has just one row and is called a row vector. The last matrix has just one column and is called a column vector. a a a. 6 a a a 3 3 33 x 4 e x, a a a, 6x 3 e 4x 3.3 5, a a a 3, Because the goal of the indexing of entries was to uniquely identify the position of an element within a matrix, one index suffices for vectors, whether they are row or column vectors. Thus, the third entry of the row vector is denoted by a 3. /7/8 Dr. Omar R Daoud 5

Matrices: Arithmetic Operations DEFINITION Two matrices A = [a jk ] and B = [b jk ] are Equal, written A = B, if and only if they have the same size and the corresponding entries are equal, that is, a = b, a = b, and so on. Otherwise, they are called Different. DEFINITION 3 o The sum of two matrices A = [a jk ] and B = [b jk ] of the same size is written as A + B and has the entries a jk + b jk obtained by adding the corresponding entries of A and B. o Matrices of different sizes cannot be added. /7/8 Dr. Omar R Daoud 6

Matrices: Arithmetic Operations DEFINITION 4 The product of any m n matrix A = [a jk ] and any scalar c (number c) is written as ca The m n matrix ca = [ca jk ] obtained by multiplying each entry of A by c. DEFINITION 5 The product C = AB (in this order) of an m n matrix A = [a jk ] times an r p matrix B = [b jk ] is defined if and only if r = n and is then the m p matrix C = [c jk ] with n entries c a b a b a b a b jk jl lk j k j k jn nk l j,, m k,, p. The condition r = n means that the second factor, B, must have as many rows as the first factor, A, has columns, namely n. A diagram of sizes that shows when matrix multiplication is possible is as follows: A B = C [m n] [n p] = [m p]. /7/8 Dr. Omar R Daoud 7

Matrices: Arithmetic Operations Introduction Rules For the addition of matrices of the same size m n: (a) A B B A (b) ( A B) C A ( B C) (written A B C) (c) A A (d) A ( A). For scalar multiplication: (a) c( A B) ca cb (b) ( c k) A ca ka (c) c( ka) ( ck) A (written cka) (d) A A. Here denotes the zero matrix (of size m n), that is, the m n matrix with all entries zero From (a) and (b), the addition is commutative and associative /7/8 Dr. Omar R Daoud 8

Matrices: Arithmetic Operations Introduction Rules For matrix multiplication: In matrix products the order of factors must always be observed very carefully. Otherwise matrix multiplication satisfies rules similar to those for numbers, namely. (a) ( ka) B k( AB) A( kb) written kab or AkB (b) A( BC) ( AB) C written ABC (c) ( A B) C AC BC Provided A, B, and C are such that the expressions on the left are defined; here, k is any scalar. (d) C( A B) CA CB From (b) the multiplication is associative, and from (c) and (d) it is distributive. /7/8 Dr. Omar R Daoud 9

Matrices: Arithmetic Operations Introduction Rules For matrix Addition: Example: A + B = C = 3 5 4 6 3 + 3 5 7 9 4 = 5 3 9 9 3 7 3 Here c = 3+ = 5, c = 5- = 3, c 3 = -+ 3 =, and so on. The entry in the box is c 3 = -6 +9 = 3. /7/8 Dr. Omar R Daoud

Matrices: Arithmetic Operations Introduction Rules For matrix Addition: Example: 3 5 4 A = 4 4 6 3 4 = 6 8 4 8 = 4 3 4 5 4 4 4 4 4 4 6 4 3 4 Here k a ka ij ij /7/8 Dr. Omar R Daoud

Matrices: Arithmetic Operations A aij Introduction Rules For matrix multiplication: Example3 : k If is an n m matrix and B b m ij C= A.B c ij ik is an a b kj c ij mq matrix then is an nq matrix where AB 3 5 3 43 4 4 5 7 8 6 6 4 6 6 3 9 4 9 4 37 8 Here c = 3 + 5 5 + ( ) 9 =, and so on. The entry in the box is c 3 = 4 3 + 7 + = 4. The product BA is not defined. /7/8 Dr. Omar R Daoud

Matrices: Arithmetic Operations Introduction Rules For matrix multiplication: Example: given A =, B = 99 99 but. 99 99 Matrix Multiplication Is Not Commutative, AB BA in General, so AB = does not necessarily imply BA = or A = or B = /7/8 Dr. Omar R Daoud 3

Matrices: Arithmetic Operations column vectors and conversely. DEFINITION 6 The transpose of an m n matrix A = [a jk ] is the n m matrix A T (read A transpose) that has the first row of A as its first column, the second row of A as its second column, and so on. Thus the transpose of A is A T = [a kj ], written out A T If A is the given matrix, then we denote its transpose by A T As a special case, Transposition converts row vectors to For square matrices, we can also reflect the elements along the main diagonal. a a a m a a a m a. kj a a a n n mn Example : Given A = 9 5, find AT A T = 9 5 /7/8 Dr. Omar R Daoud 4

Matrices: Arithmetic Operations Introduction Rules For matrix transposition: T (a) ( A ) (b) ( A B) A B (c) ca) A ca T T T (d) ( AB) B A. T T T T T T Note that in (d) the transposed matrices are in reversed order. /7/8 Dr. Omar R Daoud 5

Matrices: Arithmetic Operations DEFINITION 7 Symmetric matrices are square matrices whose transpose equals the matrix itself A T = A (thus a kj = a jk ). A = 5 = A T 5 3 DEFINITION 8 Skew-symmetric matrices are square matrices whose transpose equals the minus matrix itself. A T = -A (thus a kj = -a jk ) hence a jj =. A = 3 3 = A T /7/8 Dr. Omar R Daoud 6

Matrices: Arithmetic Operations DEFINITION 9 Upper triangular matrices are square matrices that can have nonzero entries only on and above the main diagonal, whereas any entry below the diagonal must be zero. b b b 3 B = b b 3 b 33 Lower triangular matrices can have nonzero entries only on and below the main diagonal. Any entry on the main diagonal of a triangular matrix may be zero or not. A = a a a a 3 a 3 a 33 3 4 3 9 3, 3, 8, 6 7 6 8 9 3 6 /7/8 Dr. Omar R Daoud 7

Matrices: Arithmetic Operations DEFINITION Diagonal Matrices are square matrices that can have nonzero entries only on the main diagonal. Any entry above or below the main diagonal must be zero. B = b b = b 33 5 7 Scalar Matrix is a matrix that has equal diagonal entries of a diagonal matrix S. The multiplication of any square matrix A of the same size by S has the same effect as the multiplication by a scalar c, AS = SA = ca. A unit matrix (or identity matrix) is a scalar matrix, whose entries on the main diagonal are all, and it is denoted by I n or simply by I. AI = IA = A. I /7/8 Dr. Omar R Daoud 8

Matrices: Determinants DEFINITION A determinant of second order is denoted by D and defined by a a D a a a a det A. a a A determinant of third order can be defined by a a a a a a a a a D a a a a a a 3 3 3 3 3 3 a a a a a a 3 33 3 33 3 3 3 33 a a a So here we have bars (whereas a matrix has brackets).. The signs on the right are + +.. Each of the three terms on the right is an entry in the first column of D times its minor, that is, the second-order determinant (for a delete the first row and first column, and so on). /7/8 Dr. Omar R Daoud 9

Matrices: Determinants Cramer s Rule: A determinant of order n is a scalar associated with an n n matrix A = [a jk ] is a a a n a a a n D det A. a a a m m mn For n =, this determinant is defined by D = a. For n by D = a j C j + a j C j + + a jn C jn ( j =,,, or n) or D = a k C k + a k C k + + a nk C nk (k =,,, or n) Where C jk = ( ) j+k M jk M jk is a determinant of order n, namely, the determinant of the submatrix of A obtained from A by omitting the row and column of the entry a jk, that is, the jth row and the kth column We may expand D by any row or column. /7/8 Dr. Omar R Daoud

Matrices: Determinants Cramer s Rule: Terms used in connection with determinants are taken from matrices: In D we have n entries a jk n rows and n columns, and a main diagonal on which a, a,, a nn stand. Two terms are new: M jk is called the minor of a jk in D, and C jk the cofactor of a jk in D. For later use we note that may also be written in terms of minors n jk D ( ) a M ( j,,, or n) k n jk D ( ) a M ( k,,, or n) j jk jk jk jk /7/8 Dr. Omar R Daoud

Matrices: Determinants Cramer s Rule: A minor of a matrix is the determinant of some smaller square matrix, obtained by removing one or more of its rows and columns. Given a matrix A, the minor obtained by removing the j th row and k th column is denoted by M jk. It is also called the minor of the (j, k)- entry a jk in A. Expansion on the first row: /7/8 Dr. Omar R Daoud

Matrices: Determinants Cramer s Rule: Sign Pattern: Expansion on the first row Expansion on the second row Expansion on the last row /7/8 Dr. Omar R Daoud 6

Matrices: Determinants Cramer s Rule: The minor together with the appropriate sign is called cofactor. Given a matrix The cofactor of C ij is defined as: Expansion on the i th row (i=,,3): Expansion on the j th column (j=,,3): /7/8 Dr. Omar R Daoud 7

Matrices: Determinants Cramer s Rule: Every square matrix has a determinant. The determinant of a matrix is a number. Matrix: (+) 3.5 3 3 Matrix: (-) = 3.5 =.5 = 695 /7/8 Dr. Omar R Daoud 8

Matrices: Determinants Cramer s Rule: Example: Expansion of a 3 rd order determinant 3 6 4 4 6 D 6 4 3 =( ) 3(4 + 4) + ( + 6) = This is the expansion by the first row. The expansion by the third column is D 6 3 3 4. 6 /7/8 Dr. Omar R Daoud 9

Matrices: Determinants Elementary Row Operations: An attractive way of finding determinants An upper triangular determinant is achieved, whose value is then very easy to compute, being just the product of its diagonal entries. Behavior of an nth-order Determinant under Elementary Row Operations: Interchange of two rows/columns multiplies the value of the determinant by. Addition of a multiple of a row/column to another row/column does not alter the value of the determinant. Multiplication of a row/column by a nonzero constant c multiplies the value of the determinant by c. (This holds also when c =, but no longer gives an elementary row operation.) /7/8 Dr. Omar R Daoud 3

Matrices: Determinants Elementary Row Operations: An attractive way of finding determinants An upper triangular determinant is achieved, whose value is then very easy to compute, being just the product of its diagonal entries. Behavior of an nth-order Determinant under Elementary Row Operations: Transposition leaves the value of a determinant unaltered. A zero row/column renders the value of a determinant zero. Proportional rows/columns render the value of a determinant zero. In particular, a determinant with two identical rows/columns has the value zero. /7/8 Dr. Omar R Daoud 3

Matrices: Determinants Elementary Row Operations: Rank in Terms of Determinants: Consider an m n matrix A = [a jk ]: A has rank r if and only if A has an r r submatrix with a nonzero determinant. The determinant of any square submatrix with more than r rows, contained in A (if such a matrix exists!) has a value equal to zero. Furthermore, if m = n, we have: An n n square matrix A has rank n if and only if det A. /7/8 Dr. Omar R Daoud 3

Matrices: Determinants Elementary Row Operations: Example: Evaluation of Determinants by Reduction to Triangular Form. Row Row Row 4.5 Row 4 6 5 9 6 8 3 Row 3.4 Row Row 4.6 Row 4 6 5 9.4 3.8.4 9. 5.4 47.5 34. D 4 6 4 5 6 3 8 9 Row 4 4.75 Row 3 4 6 5 9.4 3.8 47.5 /7/8 Dr. Omar R Daoud 33

Matrices: Determinants Rule of Sarrus (Diagonal method): Rewrite first two rows of the matrix, multiply diagonals going up! multiply diagonals going down! Bottom minus top! 4. 3 8 6 7 4 5 9 6 - (-5) 6 + 5 = 78 /7/8 Dr. Omar R Daoud 34

Matrices: Determinants Rule of Sarrus (Diagonal method): Rewrite first two rows of the matrix, multiply diagonals going up! multiply diagonals going down! Bottom minus top! 5 3 5 3 3 38-38 = /7/8 Dr. Omar R Daoud 35

Matrices: Determinants Rule of Sarrus (Diagonal method): Transpose has the same determinant AB = A B /7/8 Dr. Omar R Daoud 36

Matrices: Determinants Example: The determinant of a matrix product: Given Find A, B, and AB /7/8 Dr. Omar R Daoud 37 3 A 3 B 7 3 A 3 B 5 6 4 8 3 3 AB 8 4 6 77 5 AB

Matrices: Inversion DEFINITION The inverse of an n n matrix A = [a jk ] is denoted by A and is an n n matrix such that A = A A = I where I is the n n unit matrix DEFINITION 3 If A is a square matrix, and if a matrix B of the same size can be found such that AB=BA=I, then A is said to be invertible and B is called an inverse of A. If no such matrix B can be found, then A is said to be singular. /7/8 Dr. Omar R Daoud 38

Matrices: Inversion Properties of Matrix Invers: If A is an invertible matrix then its inverse is unique. (A - ) - = A. (A k ) - = (A - ) k (we will denote this as A -k ) (ca) - = (/c)a -, c. ( A T ) - = (A - ) T. If A is an invertible matrix, then the system of equations Ax = b has a unique solution given by x = A - b. A singular matrix, is the matrix that has no inverse. /7/8 Dr. Omar R Daoud 39

Matrices: Inversion DEFINITION 3 The Adjoint matrix of [A], Adj[A], is obtained by taking the transpose of the cofactor matrix of [A]. The minor for element a ij of matrix [A] is found by removing the ith row and jth column from [A] and then calculating the determinant of the remaining matrix. The matrix inverse of a matrix can be calculated by making use of the Adjoint matrix as where: A A adj( A) A is the determinant of the matrix A, adj(a) is the adjoint matrix, which can be found from the cofactor matrix /7/8 Dr. Omar R Daoud 4

Matrices: Inversion C ij = ( ) - i + j M ij Adjoint Matrix Method (Inverse of a Matrix by Determinants): -4 5 Example: Find the matrix inverse of A -3 5 7. T 5 3-8 3 5 3 5 3 5 4 5 3 4 5 3 4 5 3 adj A = [C] T = 5 7 8 5 7 8 5 7 8 5 7 M = 3 = -8 M M M 3 M M M 3 M 3 M 3 M 33-3 7 M = 5 = -8-3 5 M 3 = 5 = 3-6 - -34 The resulting matrix of minors is: M -6 - -34 7-4 6-53 9 - The resulting cofactor matrix is: C -6-34 -7-4 -6-53 -9 - /7/8 Dr. Omar R Daoud 4

Matrices: Inversion - [ A] = adj [ A] A -4 5 Example: Find the matrix inverse of A -3 5 7. 5 3-8 Adjoint Matrix Method: C -6-34 -7-4 -6-53 -9 - and then the determinant is: adj A -6-7 -53-4 -9-34 -6 - -6-7 -53-4 -9 A -336-34 -6 - -4 5 A = -3 5 7 = -336 5 3-8 A 6 7 53 336 336 336-4 9 336 336 336 7 3 68 68 68 /7/8 Dr. Omar R Daoud 4

Matrices: Inversion - [ A] = adj [ A] A Adjoint Matrix Method: Example: Find the matrix inverse of A 3. 3 4 det A = ( 7) 3 + 8 = C 7, C, C 3, 3 3 4 3 4 3 C 3, C, C 7, 3 4 4 3 3 C 8, C, C, 3 3 33 3 3 3 A.7..3.3..7..8.. /7/8 Dr. Omar R Daoud 43

Matrices: Inversion C ij = ( ) - i + j M ij A Adjoint Matrix Method: Example3: Find the matrix inverse of. 4 5 5 4 A 4 A 5 A3 6 6 3 3 A A3 A 3 6 6 3 3 A3 A3 5 A33 4 4 5 5 4 Then the inverse will be T 4 5 4 4 3 5 3 5 A 5 4 4 4 3 A 4 5 6 4 The cofactor matrix is: 4 5 4 3 5 4 6 5 3 5 /7/8 Dr. Omar R Daoud 44

Matrices: Inversion C ij = ( ) - i + j M ij Adjoint Matrix Method: 3 A, 4 Example4: Find the matrix inverse of. 4.4. A 3..3 /7/8 Dr. Omar R Daoud 45

Matrices: Inversion DEFINITION 4 If Gauss Jordan elimination method is applied on a squared matrix, thus it can be used to calculate the matrix's inverse. This can be done by augmenting the square matrix with the identity matrix of the same dimensions., and through the following matrix operations: A I A A I I A /7/8 Dr. Omar R Daoud 46

Matrices: Inversion Gauss-Jordan Elimination Method: A 3. 3 4 Example: Determine the inverse A of Row 3 Row Row Row 3 Row.5 Row Row 3 Row. Row 3 3 3.5.5.5 3 4.8.. Row Row 3 Row 3.5 Row 3 A I 7 3 7 3 5 4.6.4.4.3..7.8.. Row Row.7..3.3..7.8.. /7/8 Dr. Omar R Daoud 47

Matrices: Inversion Gauss-Jordan Elimination Method: Example: Determine the inverse A of A 3. 3 4.7..3 3.3..7. 3 4.8.. Hence AA = I. Similarly A A = I. /7/8 Dr. Omar R Daoud 48

Matrices: Inversion Gauss-Jordan Elimination Method: Example: Determine the inverse A of /7/8 Dr. Omar R Daoud 49 A I A 4 3 4 4 4 3 A I

Matrices: Rank DEFINITION 4 A positive integer r is said to be the Rank of a non-zero matrix A, if There exists at least one non-zero minor of order r of A Every minor of order greater than r of A is zero. Thus, it is the maximum number of linearly independent row vectors of A. Specifications: Any row operation does not change the column- rank. By the same argument, apply to the transpose of the matrix, we conclude that any column operation does not change the row-rank as well We call a matrix A row-equivalent to a matrix A if A can be obtained from A by (finitely many!) elementary row operations. The maximum number of linearly independent row vectors of a matrix does not change if we change the order of rows or multiply a row by a nonzero c or take a linear combination by adding a multiple of a row to another row. This shows that rank is invariant under elementary row operations. /7/8 Dr. Omar R Daoud 5

Matrices: Rank Example: Determine the rank of A= Linearly independent Linearly independent Linearly independent Linearly dependent Linearly independent Linearly independent Linearly dependent Row-Rank = /7/8 Dr. Omar R Daoud 5

Matrices: Rank Example: Determine the rank of A= Row reductions Row-rank = Row-rank = Because row reductions do not affect the number of linearly independent rows /7/8 Dr. Omar R Daoud 53

Matrices: Rank Example: Determine the rank of A= List all combinations of columns Column-Rank = Linearly independent?? Y Y Y Y Y Y Y Y Y Y N N N N N /7/8 Dr. Omar R Daoud 54

Matrices: Rank Example: Determine the rank of Row + Row Row 3 7 Row 3 4 8 58 4 9 3 A 6 4 4 54 5 Row 3 + Row. 3 4 8 58 (given) The last matrix has two nonzero rows. Hence rank A =. /7/8 Dr. Omar R Daoud 55

Matrices: Rank Example3: Determine the rank of A= Apply row reductions. row-rank and column-rank do not change. Apply column reductions. row-rank and column-rank do not change. The top-left corner is an identity matrix. The row-rank and column-rank of this normal form is certainly the size of this identity submatrix, and are therefore equal. /7/8 Dr. Omar R Daoud 56

Matrices: Rank Example4: Determine the rank of A= Rank = Rank = Rank = Rank = Rank = Rank = Rank = /7/8 Dr. Omar R Daoud 57

Matrices: Rank vs. Determinant Determinant Real number Defined to square matrix only Non-zero det implies existence of inverse. When det is zero, We know that all the columns (or rows) together are linearly dependent, but don t know any information about subset of columns (or rows) which are linearly independent. Rank Integer Defined to any rectangular matrix When applied to nn square matrix, rank=n implies existence of inverse. /7/8 Dr. Omar R Daoud 58

Matrices: Solution of Linear Systems of Equations DEFINITION 5 A linear system of m equations in n unknowns x,, x n is a set of equations of the form a x + a x.. +a n x n = b a x + a x.. +a n x n = b a m x + a m x.. +a mn x n = b m The system is called linear because each variable x j appears in the first power only, Just as in the equation of a straight line. a,, a mn are given numbers, called the coefficients of the system. b,, b m on the right are also given numbers. If all the bj are zero, then it is called a homogeneous system. If at least one bj is not zero, then it is called a nonhomogeneous system. A solution of it, is a set of numbers x,, x n that satisfies all the m equations. A solution vector is a vector x whose components form the solution. If the system is homogeneous, it always has at least the trivial solution x x n =. /7/8 Dr. Omar R Daoud 59

Matrices: Solution of Linear Systems of Equations Matrix From Linear System From the definition of matrix multiplication, the m equations may be written as a single vector equation as Ax = b, where the coefficient matrix A = [a jk ] is the m n matrix. A = a a n, x = a m a mn x x n and b = The augmented matrix of A, Ã could be formed as a a n b A = a m a mn It is merely a reminder that the last column of Ã did not come from matrix A but came from vector b The augmented matrix Ã determines the system completely because it contains all the given and needed numbers appearing in the equations. b m b b m /7/8 Dr. Omar R Daoud 6

Matrices: Solution of Linear Systems of Equations Gauss Elimination Method: Back Substitution: Step : Write the augmented matrix ( three parameters as an example (x3, x, x)) Step : Elimination of x: Call the first row of A the pivot row and the first equation the pivot equation. Call the coefficient of its x-term the pivot in this step. Use this equation to eliminate x (get rid of x) in the other equations. Step 3: Elimination of x: The first equation remains as it is. We want the new second equation to serve as the next pivot equation. But if it has no x-term (in fact, it is = ), we must first change the order of the equations and the corresponding rows of the new matrix (put = at the end and move the third equation and the fourth equation one place up). This is called partial pivoting (as opposed to the rarely used total pivoting, in which the order of the unknowns is also changed). Step 4: Back substitution: Triangular means that all the nonzero entries of the corresponding coefficient matrix lie above the diagonal and form an upside-down 9 triangle. Then we can solve the system by back substitution. Determination of x3, x, x (in this order) Working backward from the last to the first equation of this triangular system, we can now readily find x3, then x, and then x /7/8 Dr. Omar R Daoud 6

Matrices: Solution of Linear Systems of Equations Gauss Elimination Method: Back Substitution: Example: Solve the following linear system: Augmented Matrix Ã Elimination of x Row Row Row 4 Row 5 9 3 8 5 9 8 Elimination of x Row 3 3 Row 5 9 95 9 Elimination of x step: Move the third equation and the fourth equation from the previous step one place up x x x 3 x x x 3 x 5x 9 3 x x 8. Back Substitution /7/8 Dr. Omar R Daoud 6 3 3 3 95x 9 x 5 x 9 x x x.

Matrices: Solution of Linear Systems of Equations Gauss Elimination Method: Back Substitution: 6x x +x 3 +4x 4 = x Example: Solve the following linear system: 8x +6x 3 +x 4 = 34 3x 3x +9x 3 +9x 4 = 7 6x +4x +x 3 8x 4 = 38 /7/8 Dr. Omar R Daoud 63

Matrices: Solution of Linear Systems of Equations Gauss Elimination Method: Back Substitution: Example: Solve the following linear system: 6x x +x 3 +4x 4 = x 8x +6x 3 +x 4 = 34 3x 3x +9x 3 +9x 4 = 7 6x +4x +x 3 8x 4 = 38 /7/8 Dr. Omar R Daoud 64

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: First, suppose that Gaussian elimination can be performed on the system Ax = b without row interchanges having nonzero pivot elements a ii (i) for each i =,,.. n. For j =,3,.. n perform the operation E j m j, E E j, where m j, = a j () a (), these operations transform the system into one in which all the entries in the first column below the diagonal are zeros. This is called the first Gaussian transformation matrix. /7/8 Dr. Omar R Daoud 66

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: M () Ax = b = A () x = b () In similar manner we construct M () the identity matrix with the entries below the diagonal in the second column replaced by the negatives of the multipliers, m j, = a j () a () The product of M() by A () has zeros below the diagonal in the first two columns. M () A () x = b () = M () M () ( Ax = b) = A (3) x = b (3) In general, with A (k) x = b (k) and then A (k+) x = M (k) A (k) x = M (k) M () Ax = M (k) b k = b (k+) = M (k) M () b /7/8 Dr. Omar R Daoud 67

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: The process ends with the formation of A (n) x = b (n), where A (n) = M (n ) M (n ) M A is an upper triangular matrix This process forms the upper portion of the matrix factorization, U = A (n). To determine the complementary lower triangular matrix, L = L () L () L (n ), where L (k) (k) = M /7/8 Dr. Omar R Daoud 68

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: (LU Factorization Algorithm) To factor an n n matrix A = LU = a ij = l ij u ij follow Step : select l ij and u ij satisfying a ij = l ij u ij Step : for j =,, n set u j = a j l l j = a j u as the first row of U as the first column of L Step 3: for i =,, n Do i Step 3: select l ii and u ii satisfying l ii u ii = a ii k= l ik u ki Step 3: for j = +,., n set u ij = l i ii a ij k= l ik u kj for the i-th row of U l ji = u i ii a ji k= l jk u ki for the j-th column of L Step 4: select l nn and u nn satisfying l nn u nn = a nn k= l nk u kn Step 5: Output l ij for j =,.., i and i =,.., n Output u ij for j = i,.., n and i =,.., n n From Step 4: If l nn u nn =, then A = LU but A is singular /7/8 Dr. Omar R Daoud 69

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: Thus, the LU construction will be LU = L () L () L (n ) A (n) = M () M () M (n ) M (n ) M (n ) M A = A x x +3x 4 = 8 x Example: Solve the following linear systems of equations x x 3 +x 4 = 7 3x x x 3 +x 4 = 4 x +x +3x 3 x 4 = 7 Part : Find the Upper Matrix Step : A = 3 3 3, M () = 3 Step : A () = M () A = 4 3 3 5 7 3 m j, = a j () a () /7/8 Dr. Omar R Daoud 7

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: x x +3x 4 = 8 x Example: Solve the following linear systems of equations x x 3 +x 4 = 7 3x x x 3 +x 4 = 4 x +x +3x 3 x 4 = 7 Step 3:A (3) = M () A () = Part : Find the Lower Matrix 4 3 4 3 3 5 7 3 = 3 5 3 3 3 Step : M () = 3, M () = 4 3 Step: L = M () M () = 3 4 3 /7/8 Dr. Omar R Daoud 7

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: x x +3x 4 = 8 x Example: Solve the following linear systems of equations x x 3 +x 4 = 7 3x x x 3 +x 4 = 4 x +x +3x 3 x 4 = 7 Part 3: The Solution Step : Ax = LUx = b = Step: Suppose y = Ux b = Ly 3 4 3 3 5 3 3 3 x x x 3 x 4 = 8 7 4 7 3 4 3 y y y 3 y 4 = 8 7 4 7 y = 8 y y = 7 y = 9 3y + 4y + y 3 = 4 y 3 = 6 y 3y + y 4 = 7 y 4 = 6 /7/8 Dr. Omar R Daoud 7

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: x x +3x 4 = 8 x Example: Solve the following linear systems of equations x x 3 +x 4 = 7 3x x x 3 +x 4 = 4 x +x +3x 3 x 4 = 7 Part 3: The Solution Step 3: Ux = y = 3 5 3 3 3 x x x 3 x 4 = 8 9 6 6 x + x + 3x 4 = 8 x = 3 x x 3 5x 4 = 9 x = 3x 3 + 3x 4 = 6 x 3 = 3x 4 = 6 x 4 = /7/8 Dr. Omar R Daoud 73

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: 6x x +x 3 +4x 4 = x Example: Solve the following linear systems of equations 8x +6x 3 +x 4 = 34 3x 3x +9x 3 +3x 4 = 7 6x +4x +x 3 8x 4 = 38 Part : Find the Upper Matrix Step : A = 6 3 8 3 6 4 Step 3: A (3) = M () A () = 4 6 9 3, M () = 8 3 6 4, Step : A () = M () A = 4 8 3 4 = 6 4 6 4 4 5 4 4 4 8, 3 4 Step 4: A (4) = M (3) A (3) = 6 4 4 5 4 4 = 6 4 4 5 3 = U /7/8 Dr. Omar R Daoud 74

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: 6x x +x 3 +4x 4 = x Example: Solve the following linear systems of equations 8x +6x 3 +x 4 = 34 3x 3x +9x 3 +3x 4 = 7 6x +4x +x 3 8x 4 = 38 Part : Find the Lower Matrix Step : M () =, M () = 3, M (3) =, Step: L = M () M () M (3) = 3 /7/8 Dr. Omar R Daoud 75

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: 6x x +x 3 +4x 4 = x Example: Solve the following linear systems of equations 8x +6x 3 +x 4 = 34 3x 3x +9x 3 +3x 4 = 7 6x +4x +x 3 8x 4 = 38 Part 3: The Solution Step : Ax = LUx = b = 3 6 4 4 5 3 x x x 3 x 4 = 34 7 38 Step: Suppose y = Ux b = Ly 3 y y y 3 y 4 = 34 7 38 y = y + y = 34 y =. 5y + 3y + y 3 = 7 y 3 = 9 y.5y + y 3 + y 4 = 38 y 4 = 3 /7/8 Dr. Omar R Daoud 76

Matrices: Solution of Linear Systems of Equations Lower and Upper Decomposition Method: 6x x +x 3 +4x 4 = x Example: Solve the following linear systems of equations 8x +6x 3 +x 4 = 34 3x 3x +9x 3 +3x 4 = 7 6x +4x +x 3 8x 4 = 38 Part 3: The Solution Step 3: Ux = y = 6 4 4 5 3 x x x 3 x 4 = 9 3 6x x + x 3 + 4x 4 = x = 4x + x 3 + x 4 = x = 3 x 3 5x 4 = 9 x 3 = 3x 4 = 3 x 4 = /7/8 Dr. Omar R Daoud 77

Matrices: Solution of Linear Systems of Equations Cramer s Rule: Cramer s Rule: relies on determinants, Based on the coefficient matrix DEFINITION 6 ax + by = e cx + dy = f a c a x + a x.. +a n x n = b a For the linear systems of equations x + a x.. +a n x n = b with a coefficient matrix of a m x + a m x.. +a mn x n = b m a a n A =, with a det A the solution can be found based on Cramer s rule as a m a mn b d x = b a a n b m a m a mn det (A), x = a b a n a m b m a mn det (A),.., x n = a a b a m a m bm det (A) /7/8 Dr. Omar R Daoud 78

Matrices: Solution of Linear Systems of Equations Cramer s Rule: Example: Solve the following linear systems of equations 8 +5x = x 4x = Step : A = 8 5 4, b = Step : det A = 8 5 4 = 4 Step 3: x = 5 4 det (A) =, x = 8 det (A) = /7/8 Dr. Omar R Daoud 79

Matrices: Solution of Linear Systems of Equations Cramer s Rule: Example: Solve the following linear systems of equations x 4x + 5x 3 = 36 3x + 5x + 7x 3 = 7 5x + 3x 8x 3 = 3 Step : A = Step : Step 3: det A = 4 5 3 5 7 5 3 8, b = 4 5 3 5 7 5 3 8 36 7 3 = 336 x = 36 4 5 7 5 7 3 3 8 det (A) =, x = 36 5 3 7 7 5 3 8 det (A) =-3, x 3 = 4 36 3 5 7 5 3 3 det (A) =4 /7/8 Dr. Omar R Daoud 8

Matrices: Solution of Linear Systems of Equations Matrix Inversion: DEFINITION 7 a x + a x.. +a n x n = b a For the linear systems of equations x + a x.. +a n x n = b with a coefficient matrix of a m x + a m x.. +a mn x n = b m a a n A =, with a det A the solution can be found based on the matrix inversion rule as a m a mn x = A b x x n = a a n a m a mn b b m /7/8 Dr. Omar R Daoud 8

Matrices: Solution of Linear Systems of Equations Matrix Inversion Rule: Example: Solve the following linear systems of equations x + x + x 3 = x + x = x + x + x 3 = 3 Step : A = Step : A = Step 3: 3 4 4 4, b = 3 x = 3 4 4 4 3 = x =, x =, x 3 = /7/8 Dr. Omar R Daoud 8

Matrices: Solution of Linear Systems of Equations Matrix Inversion Rule: Example: Solve the following linear systems of equations x + x + x 3 = x x + x 3 = 3x + x + 3x 3 = 4 Step : A = Step : 3 3, b = 4 A Does not exist We cannot use the inverse matrix method. Whenever the inverse of a matrix does not exist, we say that the matrix is singular. /7/8 Dr. Omar R Daoud 83

Matrices: Eigenvalues and Eigenvectors DEFINITION 8 If A = a a n is n n matrix, do there exist nonzero vectors x such that Ax is a scalar a n a nn Eigenvalue multiple of x. Ax x Eigenvector A: an nn matrix : a scalar (could be zero) x: a nonzero vector in R n An eigenvalue of A is a scalar such that det( IA) (Characteristic equation of A) The eigenvectors of A corresponding to are the nonzero solutions of ( IA) x /7/8 Dr. Omar R Daoud 84

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example: Find the egigenvaluse and eigenvectors for A = 3 4 Step : Characteristic equation det (A I) = ( )( 4 ) (3)( ) = + 3 + Set + 3 + to Then = 3 9 8 =, /7/8 Dr. Omar R Daoud 85

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example: Find the egigenvaluse and eigenvectors for A = 3 4 Step : Solving for the x vector () 3 4 x x = x x x x = x x x = 3x 4x = x 3x 3x =, they are dependent of a factor of 3 and then x = x ; we can represent all eigenvectors for eigenvalue - as multiples of a simple basis vector: P = t, where t is a parameter /7/8 Dr. Omar R Daoud 86

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example: Find the egigenvaluse and eigenvectors for A = 3 4 Step : Solving for the x vector () 3 4 x x = x x x x = x 3x x = 3x 4x = x 3x x =, they are dependent of a factor of 3 and then x = 3 x ; we can represent all eigenvectors for eigenvalue - as multiples of a simple basis vector: P = t 3 = t 3, where t is a parameter /7/8 Dr. Omar R Daoud 87

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example: Find the egigenvaluse and eigenvectors for A 5 Step : Characteristic equation det( I A) 5, 3 ( )( ) Eigenvalue:, /7/8 Dr. Omar R Daoud 88

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example: Find the egigenvaluse and eigenvectors for Step : Solving for the x vector () 3 x 4 ( I A) x x 3 G.-J. E. 4 4 x 4t 4, x t t t A () 5 4 x 3 ( I A) x x 4 G.-J. E. 3 3 x 3s 3, x s s s /7/8 Dr. Omar R Daoud 89

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example3: Find the egigenvaluse and eigenvectors for Step : Characteristic equation A 5 3 I 5 A 3 Eigenvalue: ( ) ( )( 3),, 3 3 /7/8 Dr. Omar R Daoud 9

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example3: Find the egigenvaluse and eigenvectors for Step : Solving for the x vector A 5 3 () x 5 x ( I A) x x 3 x4, is a basis for the eigenspace corresponding to x t x s s t, s, t x 3 t x4 t G.-J.E. The dimension of the eigenspace of λ = is /7/8 Dr. Omar R Daoud 9

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example3: Find the egigenvaluse and eigenvectors for Step : Solving for the x vector () x 5 x ( I A) x x 3 x4 A G.-J.E. 5 x x 5t 5 t, t x 3 t x4 3 5 is a basis for the eigenspace corresponding to The dimension of the eigenspace of λ = is /7/8 Dr. Omar R Daoud 9

Matrices: Eigenvalues and Eigenvectors Finding Eigenvalues and Eigenvectors Example3: Find the egigenvaluse and eigenvectors for Step : Solving for the x vector (3) 3 3 x 5 x ( 3I A) x x 3 x4 5 is a basis for the eigenspace corresponding to 3 3 A /7/8 Dr. Omar R Daoud 93 G.-J.E. 5 3 x x 5t 5 t, t x 3 x4 t The dimension of the eigenspace of λ 3 = 3 is

Matrices: Eigenvalues and Eigenvectors DEFINITION 9 If A is an n n triangular matrix, then its eigenvalues are the entries on its main diagonal Example: Find the eigenvaluse for (a) A 5 3 3 (a) I A ( )( )( 3) 5 3 3,, 3 3 /7/8 Dr. Omar R Daoud 94

Matrices: Eigenvalues and Eigenvectors DEFINITION 9 If A is an n n triangular matrix, then its eigenvalues are the entries on its main diagonal Example: Find the eigenvaluse for (b) A 4 3 (b),,, 4, 3 3 4 5 /7/8 Dr. Omar R Daoud 95

Matrices: Diagonalization DEFINITION A squared matrix A is called Diagonalizable if there exits an invertible matrix P such that P AP is a diagonal matrix. If A and B are similar nn matrices, then they have the same eigenvalues. An nn matrix A is diagonalizable if and only if it has n linearly independent eigenvectors Example: check whether the following matrix is diagonalizable 3 3 A 3 I A 3 ( 4)( ) The eigenvalues : 4,, /7/8 Dr. Omar 3 R Daoud 96

Matrices: Diagonalization Example: check whether the following matrix is diagonalizable 3 A 3 () 4 the eigenvector p () the eigenvector p, 3 p 4 P [ p p p3], and P AP /7/8 Dr. Omar R Daoud 97

Matrices: Diagonalization Example: Show that the following matrix is not diagonalizable A = I A ( ) The eigenvalue, and then solve ( I A) x for eigenvectors I A I A eigenvector p Since A does not have two linearly independent eigenvectors, A is not diagonalizable /7/8 Dr. Omar R Daoud 98

Matrices: Diagonalization Example3: Find a matrix P such that P AP is diagonal for A = I A 3 ( )( )( 3) 3 3 3 The eigenvalues :,, 3 3 x G.-J. E. I A x 3 3 x 3 x t x eigenvector p x 3 t /7/8 Dr. Omar R Daoud 99

Matrices: Diagonalization Example3: Find a matrix P such that P AP is diagonal for A = 3 3 3 4 x G.-J. E. I A 5 4 x 3 x 3 x 4 t x 4 t eigenvector p x 3 t 4 /7/8 Dr. Omar R Daoud

Matrices: Diagonalization Example3: Find a matrix P such that P AP is diagonal for A = 3 3 3 3 x G.-J. E. 3I A x 3 4 x 3 x t x t eigenvector p 3 x 3 t /7/8 Dr. Omar R Daoud

Matrices: Diagonalization Example3: Find a matrix P such that P AP is diagonal for A = 3 3 P [ p p p3] and it follows that 4 P AP 3 A k based on the diagonalization technique A k = PD k P = λ k k λ n /7/8 Dr. Omar R Daoud

Matrices: Diagonalization Example4: Determine whether the matrix A is diagonalizable A 3 Because A is a triangular matrix, its eigenvalues are,, 3 3 The three values are distinct; Thus, A is diagonalizable /7/8 Dr. Omar R Daoud 3

Matrices: Diagonalization DEFINITION If A is nn symmetric matrix, then : A is diagonalizable (it has n linearly independent eigenvectors), except the ones in the form of A = ai All eigenvalues of A are real If is an eigenvalue of A with the multiplicity to be k, then has k linearly independent eigenvectors. That is the eigenspace of has dimension k. Any two eigenvectors from different eigenspace are orthogonal DEFINITION An nn P is orthogonal matrix if and only if its column vectors form an orthonormal set( this means that in any column vector P = [p, p,. p n ], p i p j = for i j and p i p i = Thus, a squared matrix P is called orthogonal if it is invertible and P = P T or PP T = P T P = I /7/8 Dr. Omar R Daoud 4

Matrices: Diagonalization DEFINITION If matrix A is orthogonally diagonalizable If A is a symmetric matrix If there are an orthogonal matrix P with P = P T and a diagonal matrix D such that A = PDP T = PDP A T = (PDP T ) T = P TT D T P T = A, so A is symmetric /7/8 Dr. Omar R Daoud 5

Matrices: Diagonalization Example: show that P is an orthogonal matrix P = T PP 3 3 3 3 5 3 5 4 5 5 3 I 5 3 5 4 5 5 3 5 3 5 3 5 3 3 5 3 3 3 Moreover, let p, p 5, and p 5 3, 4 5 3 5 3 5 3 5 we can produce p p p p /7/8 Dr. Omar R Daoud 6 3 3 3 5 3 5 3 3 3 5 4 3 5 p p p p p p and p p 3 5 3 5 So {p, p, p 3 } is an orthonormal

Matrices: Diagonalization Example: Find an orthogonal matrix P that diagonalizes A = 4 4 () I A ( 3) ( 6) 6, 3 (has a multiplicity of ) v () 6, v (,, ) u (,, ) 3 3 3 v (3) 3, v (,, ), v (, 4, 5) 3 v v = v v 3 = u orthogonal v v (,, ), u (,, ) 3 4 5 5 5 3 3 5 3 5 3 5 v v3 /7/8 Dr. Omar R Daoud 7

Matrices: Diagonalization /7/8 Dr. Omar R Daoud 8 Example: Find an orthogonal matrix P that diagonalizes A = 4 4 5 3 5 3 5 3 4 5 3 5 3 5 3 P 3 u u u 3 3 6 AP P

Matrices: Diagonalization Example3: If possible diagonalize the matrix A The normalized (unit) eigenvectors Then A = PDP, since P is squared and has orthonormal matrix, then it is simply orthogonal and P = P T /7/8 Dr. Omar R Daoud 9

Matrices: Diagonalization Example3: Orthogonally diagonalize the matrix A Althogh v and v are linearly independent, they are not orthogonal. Thus: Gram-Schmidt Process: The normalized (unit) eigenvectors for =7 /7/8 Dr. Omar R Daoud

Matrices: Diagonalization Example3: Orthogonally diagonalize the matrix A Gram-Schmidt Process: v i+ = u i+ i (u i+,v k ) k= v (v k,v k ) k Example: find the orthogonal version of u = orthonormal basis, u =, u 3 = and then the v = u =, v = u ( (u,v ) ) v (v,v ) u, v = + + = v, v = + + = 3 = 3 3 3 v 3 = u 3 ( (u 3,v ) (v,v ) ) v ( (u 3,v ) (v,v ) )v u 3, v = + + = v, v = + + = 3 u 3, v = 3 + 3 + 3 = 3 = v, v = 3 3 + 3 3 + 3 3 = 3 v, v, v 3 are the orthogonal basis /7/8 Dr. Omar R Daoud

Matrices: Diagonalization Example3: Orthogonally diagonalize the matrix A Gram-Schmidt Process: v i+ = u i+ i (u i+,v k ) k= v (v k,v k ) k Example: find the orthogonal version of u = orthonormal basis, u =, u 3 = and then the The present magnitude of v is + + = 3 so we replace the vector by 3 v The magnitude of v is 4 + + = 6 so we replace the vector by v 9 9 9 3 6 3 The magnitude of v 3 is + 4 + 4 = so we replace the vector by v 3 { 3 3 3 3, 3 3 6 6 6, } are the orthonormal basis /7/8 Dr. Omar R Daoud

Matrices: Diagonalization Example3: Orthogonally diagonalize the matrix A The normalized (unit) eigenvectors for =- u 3 is orthogonal to the other eigenvectors u and u. Hence {u, u, u 3 } Is an orthonormal set Then A = PDP, since P is squared and has orthonormal matrix, then it is simply orthogonal and P = P T /7/8 Dr. Omar R Daoud 3