Introduction to Matrices and Linear Systems Ch. 3

Introduction to Matrices and Linear Systems Ch. 3 Doreen De Leon Department of Mathematics, California State University, Fresno June, 5 Basic Matrix Concepts and Operations Section 3.4. Basic Matrix Concepts Definition: A matrix is an array of numbers. Notation: a a a n a a a n A = (a ij ) =...... a m a m a mn A is an m n matrix, where m is the number of rows and n is the number of columns. If m = n, A is called a square matrix. If A is a square matrix, the diagonal of A is the set of entries of the form a ii. The diagonal is only defined from the top left to the bottom right of the matrix. Applications Incidence Matrices: Matrices can be used to characterize connections in electrical networks, in nets of roads, in production processes, etc. as follows.. Nodal Incidence Matrix. The newtwork below consists of six branches (connections) and four nodes (points where two or more branches come toghether). One node is the reference node (gournded node, whose voltage is zero). We number the other nodes and number and direct the branches. This is done arbitrarily. The network can now be described by a matrix A = (a jk ), where + if branch k leaves node j a jk = if branch k enters node j if branch k does not touch node j.

Example: A is called the nodal incidence matrix of the network. For the network in the example, A =.. Mesh Incidence Matrix. A network can also be characterized by the mesh incidence matrix M = (m jk ), where + if branch k is in mesh j and has the same orientation m jk = if branch k is in mesh j and has the opposite orientation if branch k is not in mesh j, and a mesh is a loop with no branch in its interior (or in its exterior). Here, the meshes are numbered and directed (oriented) in an arbitrary fashion. We can label the meshes for the network above as follows:

Then, the mesh incidence matrix is M =. Example Coefficient Matrices: Given the system of equations 3x y + 5z = x + 4y = the coefficient matrix is ( 3 5 4 Vectors: Vectors come in two flavors, ). (column) vector a matrix having only one column, and row vector a matrix with only one row. A (column) vector has the form Example: A row vector is of the form Example: v v v =.. v = v n 5 4. 3 v = ( v v v n ). u = ( 3 4 5 ). Note: Often, a column vector is associated with a point in space and is written as, e.g., v = (5, 4, 3, ). 3

Special matrices: upper triangular a square matrix having all zeros below the diagonal. Any of the other entries may be zero. The basic form is a a a n a a n....... a nn Example: 3. lower triangular a square matrix having all zeros above the diagonal. Any of the other entries may be zero. The basic form is a a a....... a n a n a nn Example: ( ). 3 diagonal a square matrix having all zeros above and below the diagonal. Any of the entries on the diagonal may be zero. The basic form is a a....... a nn Examples: the identity matrix I (the diagonal matrix having all ones on the diagonal); the zero matrix (the matrix having only zeros). Example: the 3 3 identity matrix. 4

Example: the 3 3 zero matrix.. Matrix Operations.. Transposition Definition: The transpose of a matrix A, denoted A T, is defined by swapping the rows and columns. In mathematical notation, it is defined as a ij a ji. Examples: Definitions: v = v T = ( 3 ) 3 ( ) 3 A = A 4 T = 3 4 symmetric A matrix A is symmetric if A T = A. Example: A = 3 4 3 4. skew-symmetric A matrix A is skew-symmetric if A T = A. Example: A = 3 4 3 4... Equality of Matrices A = (a ij ) and B = (b ij ) are equal if and only if i) A and B have the same size, and ii) a ij = b ij for all i, j. 5

..3 Matrix Addition/Subtraction Matrix addition and subtraction are defined only for matrices of the same size. The sum (difference) is found by adding (subtracting) the corresponding entries. Example: A = 3 4, B = 5 6 3 + + 3 A + B = 3 + 4 + = 4 6 5 + ( ) 6 + 3 4 9..4 Scalar Multiplication Let c be a scalar and A an m n matrix. Then ca is found by multiplying each entry in A by c. (So, A is found by taking the negative of each entry.) Example: A = ( ), c = = ca = A = 3 4 ( ) = 3 4 ( ) 4 6 8 Some laws: Here, denotes the zero matrix (i.e., the matrix whose entries are all zero). A and B are matrices and c, d, and k are scalars. a) A + B = B + A b) A + = A c) A + ( A) = d) c(a + B) = ca + cb, (c + d)a = ca + da e) (ck)a = c(ka) Transposition laws i) (A + B) T = A T + B T ii) (ca) T = ca T 6

..5 Matrix Multiplication Definition: The product C = AB of an m n matrix A and an r p matrix B is defined if and only if r = n, i.e., the number of rows of B must equal the number of columns of A. C is then the m p matrix with entries n c ij = a il b lj = a i b j + a i b j + + a in b nj. l= Idea: Take the dot product of each row i in A with each column j in B and place the results in the corresponding position of C (the ij th position). Some Applications of Matrix Multiplication. Exercise for Weight Loss (requires matrix-vector product). Suppose that in a diet by exercise program, a person weighing 85 pounds burns 35 calories per hour in walking (3 mph), 5 in bicycling (3 mph), and 95 in jogging (6.6 mph). Bill, weighing 85 lb, plans to exercise the number of hours each day given in matrix A. Vector v contains the calories burned in each exercise. Thus, the total calories burned each day can be determined by finding the matrix-vector product Av. and A = W B J..5...5.5.5..5. 35 v = 5. 95 MON WED FRI SAT. Computer Production (requires matrix-matrix product). Suppose a computer production company produces two major models of computers, PC86 and PC86. The matrix A shows the cost per computer (in thousands of dollars) and matrix B shows the production figures for each quarter for the year (in multiples of, units). The product of the two matrices will show shareholders the cost per quarter (in millions of dollars) for raw material, labor, and miscellaneous. and A = PC86 PC86..6.3.4.5.6 B = Quarter 3 4 ( ) 3 8 6 9 6 4 3 7 Raw Components Labor Miscellaneous PC86 PC86.

Examples: ( ) 5 ) A =, v = 3 4 6 3 ( ) + + 5 3 Av = 3 + 4 + 6 3 ( ) = 9 Note: va is not defined. 3 ) A = 4 5 6, B = 3 3 AB = 4 5 6 3 () + ( ) + 3() () + ( ) + 3() = 4() + 5( ) + 6() 4() + 5( ) + 6() () + ( )( ) + ( 3)() () + ( )( ) + ( 3)() 4 = 5 4 Note: BA is not defined. WARNING: Note that ) AB BA in general. ) AB = does not necessarily imply (a) A = (b) B = (c) BA = 3) AC = AD does not necessarily imply C = D. Other properties of matrix products: In the following, A, B, and C are matrices and k is a scalar. i) (ka)b = k(ab) 8

ii) A(BC) = (AB)C iii) (A + B)C = AC + BC iv) C(A + B) = CA + CB v) (AB) T = B T A T.3 Linear Systems of Equations and Gaussian Elimination Section 3. Linear systems of equations appear in many applications, e.g., analysis of circuits using Kirchhoff s laws. The goal in this section is to solve linear systems of equations using Gaussian elimination, a method closely related to the standard method of elimination that you probably learned in high school. The difference is that Gaussian elimination involves using matrices and systematically modifying them to obtain a system directly solvable by back substitution (solving the last equation for the last variable and working our way up to the first equation, substituting the solved variables into the remaining equations). Idea: Write a linear system of equations in matrix form. Let A = the coefficient matrix, a x + a x + + a n x n = b a x + a x + + a n x n = b.... a m x + a m x + + a mn x n = b m x x x n b b b m x =., and b =. () Then we can rewrite () in the form Example: Ax = b. Given the system of equations we have 3x + x x 3 = x + x 3 = x + x 3 = x A =, x = x, and b =. x 3 9

Multiply to verify that Ax = b is equivalent to the system given. To solve the system in (), we will first define the augmented matrix a a a n b a a a n b à = (A b) =.... a m a m a mn b m Then, we will use Gaussian elimination to solve: Gaussian elimination uses elementary row operations to reduce à to (row) echelon form; then back-substitution is used to solve the system. First, we will define echelon form and then identify the elementary row operations. The elementary row operations are directly related to the procedures used to solve linear systems of equations by the method of elimination that you probably learned in high school. Definition: A matrix is in (row) echelon form if it satisfies the following properties. Every row consisting entirely of zeros lies beneath every row that contains a nonzero element. In each row with a nonzero element, the first nonzero entry (called the leading entry) lies to the right of the leading entry in every preceding row. There are three elementary row operations. They are Multiply one row by a nonzero constant. Exchange two rows. Add a nonzero constant multiple of one row to another. Definition: Matrix A is row-equivalent to matrix B if B can be obtained from A by a sequence of elementary row operations. We care about this because two systems of equations Ax = b and Bx = d have the same solution if their augmented matrices are row-equivalent. Examples: ) Solve x + x x 3 = x + x + x 3 = 3x + 5x x 3 =

r r r 3 3 r r 3 r 3 r 3 3r 3 5 3 3 r 3 r 3 r Using back-substitution, we see that x 3 = x + x 3 = = x = x 3 = 3 x + x x 3 = = x = x + x 3 = 4 In vector form, we write 4 x = 3. ) Solve ( 3 4 5 ) 6 3 4 7 3x + 4x + 5x 3 = 6 x + 3x + 4x 3 = 7 ( 3 4 5 ) 6 3 3 3 r r 3 r = x + x 3 = 9 3x + 4x + 5x 3 = 6 Since we have no leading entry for x 3, x 3 is arbitrary. So, ( r 3r 3 4 5 6 9 ) In vector form, 3) Solve Let x 3 = r = x = 9 r = x = 6 4x 5x 3 6 4(9 r) 5r = 3 3 3 + 3r = = + r 3 + r x = 9 r = r 9 + r x x + 3x 3 = 8 x + x + x 3 = 5 5x + 5x + 6x 3 = 3.

3 8 5 5 5 r r 3 8 r r r 5 r 3 r 3 5r 5 5 6 3 5 5 6 3 5 5 r 3 r 3 r 5 This leaves us with 5x + x 3 = x + x + x 3 = 5 Again, we have no leading entry for x 3, so x 3 is arbitrary. In vector form, Let x 3 = r = x = x 3 + = r + 5 5 = x = 5 x x 3 ( ) r + = 5 r 5 7r = 5 7r 5 x = r+ = 5 r 5 + r 5 7 5 5. 4) Solve x + x 3x 3 = 4 3x + x + x 3 = 8 x x + 4x 3 = 8 3 4 3 4 3 4 3 8 r 3r 5 4 3 r 3 r 5 4 r 3 r 3 r 4 8 5 4 The last row implies = 4, which is false. Therefore, there is no solution..4 Reduced Row Echelon Matrices and Gauss-Jordan Elimination Section 3.3.4. Gauss-Jordan Elimination Definition: A matrix is in reduced row echelon form if it has the properties of a matrix in row echelon form and Each leading entry is.

Each leading entry is the only nonzero element in the column. Examples: The following matrices are in reduced row echelon form. 3,, and 5 The goal of Gauss-Jordan elimination is to transform a matrix to reduced row echelon form. Theorem. Every matrix is row-equivalent to one and only one reduced row echelon matrix. The advantage of reduced row echelon form: You can easily see the number and type of solutions of a system of equations. Theorem. Every linear system either has a unique solution, no solution, or infinitely many solutions..4. Homogeneous Systems Definition: A linear system of equations is homogeneous if the right hand side is all zeros. In other words, we have a system of the form a x + a x + + a n x n = a x + a x + + a n x n =.... a m x + a m x + + a mn x n = Note: This system of equations always has the trivial solution (x =, x =,... x n = ). Therefore, homogeneous systems either have a unique solution or infinitely many solutions. Examples: ) Solve using Gauss-Jordan elimination x + x x 3 = x + x + x 3 = 3x + 5x x 3 = r r r 3 r r 3 r 3 r 3 3r 3 5 3 r r r 3 r 3 +r r r +r 3 r r r r r r 3 3 3

Solving, x 3 = x = x = ) Solve using Gauss-Jordan elimination 3 5 r r x + x x 3 = x + x = 3x + 5x x 3 = r r 3 r 3 r 3 3r r r r r 3 r 3 +r r r r This matrix is in reduced row-echelon form. Since there is no leading entry for x 3, x 3 is arbitrary and Let x 3 = r = x = x 3 = r = x = x 3 = r So, x x = x = x 3 r r r = r Theorem 3. The homogoneous system with n n coefficient matrix A has only the trivial solution if and only if A is row-equivalent to the identity matrix..5 Inverse of a Matrix Section 3.5 Motivating Example - Cryptography. Cryptography is the process of encoding and decoding messages. One type of code that is difficult to break makes use of a large invertible matrix to encode a message, called the encoding matrix. The receiver of the message decodes it using the inverse of the encoding matrix. The inverse of the encoding matrix is called the decoding matrix. We can illustrate this method using a 3 3 matrix for simplicity. Suppose that the message is STUDY MATH MORE, 4

and the encoding matrix is 3 3 4. 4 3 4 We assign a number to each letter of the alphabet. For convenience, we will associate each letter with its position in the alphabet; so, A is, B is, etc. Let a space between words be denoted by the number 7. The digital form of our message is then S T U D Y M A T H M O R E 9 4 5 7 3 8 7 3 5 8 5 Since we are using a 3 3 matrix to encode the message, we will break the digital message up into a sequence of 3 column vectors. Then, we put the message into code by multiplying each of the vectors by the encoding matrix. This is most easily done by putting the column vectors into a single matrix and multiplying: 3 3 4 9 4 3 8 5 95 57 9 5 7 8 = 4 5 4 3. 4 3 4 7 3 5 99 35 65 34 The columns of this matrix give the encoded message, which is transmitted in the following form: 4, 4,, 47, 5, 99, 43,, 35, 97, 4, 65, 4, 3, 34. To decode the message, the receiver writes this string as a sequence of 3 column vectors and repeats the above procedure using the inverse of the encoding matrix, which in this case is 4 4 3. 4 3 3 Thus, to decode the message, compute 95 57 9 9 4 3 8 5 4 4 3 4 5 4 3 = 5 7 8, 4 3 3 99 35 65 34 7 3 5 which you can recognize as the original matrix formed before the message was encoded. 5

The Matrix Inverse Recall that the n n identity matrix is the diagonal matrix having ones on the diagonal, I =.... Note that AI = IA = A. Definition: The square matrix A is invertible if there exists a matrix B such that B is an inverse matrix of A. AB = BA = I. Theorem 4. If A is invertible, its inverse is unique. The inverse is denoted by A. Properties of the Matrix Inverse () (A ) = A () (A n ) = (A ) n (3) (AB) = B A How to find A The procedure for finding the inverse of a matrix will be illustrated by example. Example: Find the inverse of A =. 3 5 r r r r 3 r 3 r 3 3r 3 5 3 5 3 r r 3 3 r r 3 3 3 5 9 4 r r r 3 r r 3 7 3 r 3 r 3 +r r r +r 3 4 4 9 4 A = 7 3. 4 6

Matrix inverses can be used to solve linear systems of equations. Suppose we wish to solve the system of equaitons Ax = b, where A is invertible. Then, For example, if we have Ax = b = A (Ax) = A b (A A)x = A b Ix = A b x = A b. Ax =, where A is the matrix given in the above example, then 4 x = A = 9. 5.6 Determinants Section 3.6 Determinants have many applications, some of which we will be studying in this class in Chapters 4 and 5, when we discuss linear independence of functions, and in Chapters 6 and 7, when we use determinants to find eigenvalues and eigenvectors. Recall: ( ) a b A = = det(a) = c d a b c d = ad bc a a a 3 A = a a a 3 = det(a) = a 3 a 3 a 33 a a a 3 a a a 3 a 3 a 3 a 33 a a a a a 3 a 3 = a a a 33 + a a 3 a 3 + a 3 a a 3 a 3 a a 3 a 3 a 3 a a 33 a a Determinants of Larger Matrices Definition: Let A = (a ij ) be an n n matrix. The ijth minor of A, denoted M ij, is the determinant of the (n ) (n ) matrix formed from deleting the ith row and jth column of A. The ijth cofactor of A, denoted A ij is defined by A ij = ( ) i+j M ij. 7

Example: Given A = M = M 3 = 4 3 4 4 3 4, = + 4 = = A = ( ) + ( ) = = + 3 4 + 3 = 3 = A 3 = ( ) +3 (3) = 3, etc. We can find the determinant of a square matrix using a cofactor expansion. Example: Using a cofactor expansion along the first row, we obtain det(a) = a A + a A + + a n A n. Note: We can use any row or column for this purpose. So, or deta = a i A i + a i A i + + a in A in (cofactor expansion along the ith row) deta = a j A j + a j A j + + a nj A nj (cofactor expansion along the jth column) Idea: Pick the row or column that makes things easiest (e.g., that has the most zeros). Example: Find the determinant of A = 4 3 4 For this matrix, it is easiest to use the cofactor expansion along the second or fourth column. We choose the fourth column. 4 deta = ( ) +4 3 4 + ( )+4 3 4 + + ( )4+4 4 3 = ( + 6 + 4) + ( + 8 + 6 8 + + 3) + ( + 4 4 4) = + 6 = 6 Some Properties of Determinants () det(a ) = det(a) 8

() det(a T ) = det(a) (3) det(ab) = det(a)det(b) Elementary Row Operations and the Determinant The determinant can also be found by using elementary row operations. First, we need the following theorem, and then we need to discuss how the elementary row operations impact the determinant. Theorem 5. The determinant of an upper (or lower) triangular matrix is the product of its diagonal elements. 3 5 7 Example: Find det(a) for A = 5 6 4 3. 3 det(a) = 3 5 7 5 6 4 3 3 = ( )(4)( 3) = 48. If we use elementary row operations to reduce a matrix to row echelon form, we can use this theorem to find its determinant. The following describe how the elementary row operations affect the determinant. () If B is obtained by multiplying a row (or column) of A by a constant k, then det(b) = kdet(a). () If B is obtained by interchanging two rows (or columns) of A, then det(b) = det(a). (3) If B is obtained by adding a constant multiple of one row (or column) of A to another row (or column) of A, then det(b) = det(a). Note: If two rows (or columns) of A are identical, then det(a) =. Note: We can use the determinant to decide if a matrix is invertible. Theorem 6. A matrix A is invertible if and only if its determinant is nonzero. So, if the determinant of a matrix is zero, the matrix is not invertible. Definition: If det(a), then A is nonsingular; otherwise, A is singular. 9

Example: Find det(a) for A in the cofactor expansion example by using the method of elimination (i.e., use elementary row operations to transform A to a triangular matrix). r r r 4 r 3 r 3 3r r 3 r 3 r r 4 r 4 4r 3 = r 4 r 4 5 r 4 8 3 = 8 4 5 7 3 7 r 4 r = 4 r 4 r 4 + 7 4 r 3 8 = 8 4 3 4 = ( ()()( 8) 3 ) 4 = 6