Chapter Basic Linear lgebra to accompany Introduction to Mathematical Programming Operations Research, Volume, th edition, by Wayne L. Winston and Munirpallam Venkataramanan Presentation by: H. Sarper Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc. Chapter - Learning Objectives Describe matrices and vectors with basic matrix operations. Describe matrices and their application to modeling systems of linear equations. Explain the application of the Gauss-Jordan method of solving systems of linear equations. Explain the concepts of linearly independent set of vectors, linearly dependent set of vectors, and rank of a matrix. Describe a method of computing the inverse of a matrix. Describe a method of computing the determinant of a matrix. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. - Matrices and Vectors Matrix an rectangular array of numbers ( ) Typical m x n matrix having m rows and n columns. We refer to m x n as the order of the matrix. The number in the ith row and jth column of is called the ijth element of and is written a ij. a a a m a a a m a n a n a mn Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. - Matrices and Vectors Two matrices = [a ij ] and B = [b ij ] are equal if and only if and B are the same order and for all i and j, a ij = b ij. If and B x w y z = B if and only if x =, y =, w =, and z = Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. - Matrices and Vectors Vectors any matrix with only one column is a column vector. The number of rows in a column vector is the dimension of the column vector. n example of a X matrix or a two-dimensional column vector is shown to the right. R m will denote the set all m-dimensional m column vectors ny matrix with only one row (a X n matrix) is a row vector.. The dimension of a row vector is the number of columns. ( ) Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. - Matrices and Vectors Vectors appear in boldface type: for instance vector v. ny m-dimensional vector (either row or column) in which all the elements equal zero is called a zero vector (written ). Examples are shown to the right. ( ) Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. - Matrices and Vectors ny m-dimensional m vector corresponds to a directed line segment in the m-dimensional m plane. For example, the two- dimensional vector u corresponds to the line segment joining the point (,) to the point (,) X (, ) X The directed line segments (vectors u, v, w) ) are shown on the figure to the right. (-, -) u v w (, -) Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. - Matrices and Vectors The scalar product is the result of multiplying two vectors where one vector is a column vector and the other is a row vector. For the scalar product to be defined, the dimensions of both vectors must be the same. The scalar product of u and v is written: ( ) ( u v ) u v u v ( ) + + + u n v n Example: u ( ) v uv ( ) + ( ) + ( ) 8 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. - Matrices and Vectors Scalar multiple of a matrix: Example addition of two matrices (of same order): C + B i, j i, j i, j B C + 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. - Matrices and Vectors ddition of vectors (of same degree). Vectors may be added using the parallelogram law or by using matrix addition. v ( ) u ( ) X (,) u u+v (,) (,) u + v ( + + ) u + v ( ) v X Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. - Matrices and Vectors Line Segments can be defined using scalar multiplication and the addition of matrices. If u = (,) and v = (,), the line segment joining u and v (called uv) is the set of all points corresponding to the vectors cu +(-c)v where < c < X u c= c=/ c= v X See the example to the right. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. - Matrices and Vectors Transpose of a matrix Given any m x n matrix a a a a T a a a a a a a a a a a a a a Example Observe: T ( T ) T Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. - Matrices and Vectors Matrix multiplication Given to matrices and B, the matrix product of and B (written B) is defined if and only if the number of columns in = the number of rows in B. The matrix product C = B of and B is the m x n matrix C whose ij th element is determined as follows: C ij = scalar product of row i of x column j of B Example C Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc. B 8 C ( ) C ( ) 8 C ( ) C ( ) Note: Matrix C will have the same number of rows as and the same number of columns as B.. - Matrices and Vectors Matrix Multiplication with Excel Use the EXCEL MMULT function to multiply the matrices: B Step Enter matrix into cells B:D and matrix B into cells B:C. Step Select the output range (B8:C9) into which the product will be computed. Step In the upper left-hand corner (B8) of this selected output range type the formula: = MMULT(B:D,B:C). Step - Press CONTROL SHIFT ENTER (not just enter) B C D Matrix - Matrix B 8 B = 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. Matrices and Systems of Linear Equations Consider a system of linear equations shown to the right. The variables (unknowns) are referred to as x, x,, x n while the a ij s and b j s are constants. set of such equations is called a linear system of m equations in n variables. a x + a x + + a n x n b a x + a x + + a n x n b a m x + a m x + + a mn x n b m solution to a linear set of m equations in n unknowns is a set of values for the unknowns that satisfies each of the system s s m equations. Example Given x is a solution to the system since (using substitution) where x x x + x x x + ( ) ( ) Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. Matrices and Systems of Linear Equations Matrices can simplify and compactly represent a system of linear equations. a a a m a a a m a n a n a mn x x x x n b b b b m system of linear equations may be written x = b and is called it s matrix representation. The matrix multiplication (using only row of the matrix for example) confirms this representation thus: a x + a x + + a n x n b Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. Matrices and Systems of Linear Equations x = b can sometimes be abbreviated b. For example, given: x x x b b is written: Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Gauss-Jordan Method Using the Gauss-Jordan method, we can show that any system of linear equations must satisfy one of the following three cases: Case Case Case The system has no solution. The system has a unique solution. The system has an infinite number of solutions. The Gauss-Jordan method is important because many of the manipulations used in this method are used when solving linear programming problems by the simplex algorithm (see Chapter ). 8 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Gauss-Jordan Method Elementary row operations (ero). n ero transforms a given matrix into a new matrix via one of the following operations: Type ero Matrix is obtained by multiplying any row of by a nonzero scalar. Type ero Multiply any row of (say, row i) by a nonzero scalar c. For some j i, let: row j of = c*(row i of ) + row j of. Type ero Interchange any two rows of. 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Gauss-Jordan Method Finding a solution using the Gauss-Jordan method. The Gauss-Jordan method solves a linear equation system by utilizing ero s in a systematic fashion. The steps to use the Gauss-Jordan method (with an accompanying example) are shown below: Solve x = b where: b 9 Step Write the augmented matrix representation: b = 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Gauss-Jordan Method The method uses ero s to transform the left side of b into an identify matrix. The solution will be shown on the right side. See x to the right. x Step t any stage, define a current row, current column, and current entry (the entry in the current row and column). Begin with row as the current row and column as the current column, and a (a = ) as the current entry. (a b = If the current entry (a) is nonzero, use ero s s to transform the current column (column ) entries to: 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Gauss-Jordan Method Steps to accomplish this were:. Multiply row by ½ (type ero). b = 9. Replace row of b by -*(row b ) + row of b (type ero). b = 9. Replace row of b by -*(row of b) + row of b (type ero). b = 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Gauss-Jordan Method Then obtain the new current row, column, and entry by moving down one row and one column to the right and go to Step. If a (current entry) would have equaled zero, then do a type ero with the current row and any row that contains a nonzero entry in the current column. Use ero s to transform column as shown to the right. If there are no nonzero numbers in the current column, obtain a new current column and entry by moving one column to the right. Step - If the new current entry is non zero, use ero s to transform it to and the rest of the column entries to. Repeat this step until finished. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Gauss-Jordan Method Making a the current entry:. Multiply row of b by -/ (type ero). Replace row of b by -*(row of b ) + row of b (type ero).. Place row of b by *(row of b ) + row of b (type ero). b = b = b = 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Gauss-Jordan Method Repeating Step again for the another new entry (a ) and performing an additional three ero s yields the final augmented array: 9 b 9 = Special Cases: fter application of the Gauss-Jordan method, linear systems having no solution or infinite number of solutions can be recognized. No solution example: Infinite solutions example: Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Gauss-Jordan Method Basic variables and solutions to linear equation systems For any linear system, a variable that appears with a coefficient of in a single equation and a coefficient of in all other equations is called a basic variable. ny variable that is not a basic variable is called a nonbasic variable. Let BV be the set of basic variables for x=b and NBV be the set of nonbasic variables for x=b. The character of the solutions to x=b (and x=b) depends upon which of following cases occur. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Gauss-Jordan Method Case : x=b has at least one row of the form [,,, c] (c ). Then x=b (and x = b) has no solution. In the matrix to the right, row meets this Case criteria. Variables x, x, and x are basic while x is a nonbasic variable. b = Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Gauss-Jordan Method Case : Suppose Case does not apply and NBV, the set of nonbasic variables is empty. Then x=b will have a unique solution. The matrix to the right has a unique solution. The set of basic variables is x, x, and x while the NBV set is empty. b = 8 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Gauss-Jordan Method Case : Suppose Case does not apply and NBV is not empty. Then x=b (and x = b) will have an infinite number of solutions. BV = {x, x, and x } while NBV = {x and x }. b = 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Gauss-Jordan Method summary of Gauss-Jordan method is shown to the right. The end result of the Gauss-Jordan method will be one either Case, Case, or Case. Does ' b ' have a row [,,..., c ] (c < > )? x = b has no solution. Yes x = b has a unique solution. Yes No Find BV and NBV. Is NBV empty? No x = b has an infinite number of solutions. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. Linear Independence and Linear Dependence Let V = {v, v,, v k } be a set of row vectors all having the same dimension. linear combination of the vectors in V is any vector of the form c v + c v + + c k where c, c,, c k are arbitrary scalars. Example: If V = { [, ], {, ] } v v = ([ ]) [ ] = [ ] and v + v = [ ] + ([ ]) = [ ] Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. Linear Independence and Linear Dependence set of vectors is called linearly independent if the only linear combination of vectors in V that equals is the trivial linear combination (c = c = = c k = ). set of vectors is called linearly dependent if there is a nontrivial linear combination of vectors in V that adds up to. Example Show that V = {[, ], [, ]} is a linearly dependent set of vectors. Since ([, ]) ([, }) = ( ), there is a nontrivial linear combination with c = and c = - that yields. Thus V is a linear independent set of vectors. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. Linear Independence and Linear Dependence What does it mean for a set of vectors to linearly dependent? set of vectors is linearly dependent only if some vector in V can be written as a nontrivial linear combination of other vectors in V. If a set of vectors in V are linearly dependent, the vectors in V are, in some way, NOT all different vectors. By different we mean that the direction specified by any vector in V cannot be expressed by adding together multiples of other vectors in V. For example, in two dimensions, two linearly dependent vectors lie on the same line. X X v = ( ) Linearly Dependent v = ( ) X v = ( ) Linearly Independent v = ( ) X Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. Linear Independence and Linear Dependence The Rank of a Matrix: Let be any m x n matrix, and denote the rows of by r, r,, r m. Define R = {r, r,, r m }. The rank of is the number of vectors in the largest linearly independent subset of R. To find the rank of matrix, apply the Gauss-Jordan method to matrix. Let be the final result. It can be shown that the rank of = rank of. The rank of = the number of nonzero rows in. Therefore, the rank = rank = number of nonzero rows in. Given V = {[ ], [ ], [ ]} Form matrix fter the Gauss-Jordan method: ' Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. Linear Independence and Linear Dependence method of determining whether a set of vectors V = {v, v,, v m } is linearly dependent is to from a matrix whose ith row is v i. If the rank of = m, then V is a linearly independent set of vectors. If the rank < m, then V is a linearly dependent set of vectors. See the example to the right. Given V = {[ ], [ ], [ ]} Form matrix fter the Gauss-Jordan method: ' Since the rank of = which is <, the set of vectors in V are linearly dependent. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Inverse of a Matrix square matrix is any matrix that has an equal number of rows and columns. The diagonal elements of a square matrix are those elements a ij such that i = j. square matrix for which all diagonal elements are equal to and all non-diagonal elements are equal to is called an identity matrix. n identity matrix is written as I m. n example is shown to the right. I Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Inverse of a Matrix For any given m x m matrix, the m x m matrix B is the inverse of if : Some square matrices do not have inverses. If there does exist an m x m matrix B that satisfies B = B = I m, then we write: B B I m B Example B B I Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Inverse of a Matrix To see why we are interested in the inverse of a matrix, suppose we want to solve a linear system x = b that has m equations in m unknowns. Suppose that - exists. The results of multiplying both sides of the equation by - is shown to the right. This shows that knowing - enables us to find the unique solution to a square linear system of equations. The Gauss-Jordan method may be used to find - (or show that - does not exist). x b ( ) x I m x b x b b 8 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Inverse of a Matrix Using the Gauss-Jordan Method to find -. Given matrix, create the augmented matrix I. Create the augmented matrix I. I = Transform into I using ero s. Step - multiply row by ½ yielding: I = 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Inverse of a Matrix Step - replace row by (- )(row) + row yielding: I = Step - multiply row by yielding: I = Step - replace row by ( /)(row) + row yielding: I has been transformed into -. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Inverse of a Matrix Some matrices do not have inverses. Consider matrix (shown to the right). pplying the Gauss-Jordan method yields. Matrix does not have a solution since the bottom row of has zeros. This can only happen if rank <. If m X m matrix has rank < m, then - will not exist. ' Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. The Inverse of a Matrix Use matrix inverses to solve linear systems. Using appropriate matrix representation to solve the system to the right. x + x x + x x b or where: ' x x x b Thus, x =, x = is the unique solution to the system. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. The Inverse of a Matrix Inverting Matrices with Excel Use the EXCEL MINVERSE function to invert the matrix: Enter the matrix into cells B:D and select the output range (B:D was chosen) where you want - computed. In the upper left-hand corner of the output range (cell B), enter the formula: = MINVERSE(B:D) Matrix Matrix - B - - C D - - Press CONTROL SHIFT ENTER and out - is computed in the output range Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. Determinants ssociated with any square matrix is a number called the determinant of (often abbreviated as det or ). For a x matrix: a a a a det = a a a a For a x matrix: det = a Example: det = () () = - Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. Determinants If is an m x m matrix, then for any values of i and j, the ijth minor of (written ij) is the (m - ) x (m - ) submatrix of obtained by deleting row i and column j of. If then 8 9 9 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. Determinants Let be any m x m matrix. We may write as: a a a m a a a m a m a m a mm To compute det, pick any value of i ( i =,,, m) : det = (-) i+ a i (det i ) + (-) i+ a i (det i ) + + (-) i+m a im (det im ) The above formula is called the expansion of det by the cofactors of row i. Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.
. Determinants Find det given matrix : 8 9 det is expanded by using row cofactors (row or row could be used with similar results). Notice that using row : a =, a = and a = and the minors of matrix. 9 9 8 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.. Determinants Thus: det = (9) 8() = - det = (9) () = - det = (8) () = - det = (-) i+ a i (det i ) + (-) i+ a i (det i ) + (-) i+ a i (det i ) = ()()(-) + (-)()(-) + ()()(-) = - + + -9 = Note: det = makes sense since the set of row vectors forming matrix are clearly linearly dependent. 8 Copyright (c) Brooks/Cole, a division of Thomson Learning, Inc.