Matrix & Linear Algebra

Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84

Vectors Vectors Vector: A vector in n-space is an ordered list of n numbers. These numbers can be represented as either a row vector or a column vector: v 1 v = ( ) v 2 v 1 v 2... v n,v =. v n We can also think of a vector as defining a point in n-dimensional space, usually R n ; each element of the vector defines the coordinate of the point in a particular direction. Jamie Monogan (UGA) Matrix & Linear Algebra 2 / 84

Vector Arithmetic Vectors Vector Addition: Vector addition is defined for two vectors u and v iff they have the same number of elements: u + v = ( u 1 + v 1 u 2 + v 2 u n + v n ) Let u = ( 3 2 1 ), v = ( 2 0 1 ). u + v = ( 3 + 2 2 + 0 1 + 1 ) = ( 5 2 2 ) Jamie Monogan (UGA) Matrix & Linear Algebra 3 / 84

Vector Arithmetic Vectors Scalar Multiplication: The product of a scalar c and vector v is: cv = ( cv 1 cv 2... cv n ) Let u = ( 3 2 1 ), c = 6. cv = ( 6 3 6 2 6 1 ) = ( 18 12 6 ) Jamie Monogan (UGA) Matrix & Linear Algebra 4 / 84

Vectors Vector Properties Let u,v,w be vectors, c,d be constants, 0 be a vector u s.t. u i = 0. Then, assuming u, v, w, 0 conformable, Commutativity: u + v = v + u Associativity: (u + v) + w = u + (v + w) Distributivity: c(u + v) = cu + cv Scalar Distributivity: (c + d)u = cu + du Additive Identity: u + 0 = u Multiplicative Identity: 1u = u Jamie Monogan (UGA) Matrix & Linear Algebra 5 / 84

Transpose Vectors Transpose Transpose: The transpose of a vector u changes u s row/column status. Let u = ( 3 2 1 ) be 1 3. 3 Then, u = 2 is 3 1. 1 u T = u Jamie Monogan (UGA) Matrix & Linear Algebra 6 / 84

Inner Product Vectors Inner Product Inner Product:The Euclidean inner product (also called the dot product) of two vectors u and v is again defined iff they have the same number of elements u v = u 1 v 1 + u 2 v 2 + + u n v n = Let u = ( 3 2 1 ), v = ( 2 0 1 ). u v =3 2 + 2 0 + 1 1= 6 + 1 = 7. If u v = 0, the vectors are orthogonal (or perpendicular). n i=1 u i v i Jamie Monogan (UGA) Matrix & Linear Algebra 7 / 84

Inner Product Vectors Inner Product Think about the u v inner product as }{{} u v 1 k }{{} k 1 = w }{{} 1 1 Or, assume u, v both k 1 columns, then u v v T u u v = v u = ( 2 0 1 ) 3 2 1 = 7 Jamie Monogan (UGA) Matrix & Linear Algebra 8 / 84

Vectors Inner Product Inner Product Properties Commutativity: u v = v u Associativity: c(u v) = (cu) v = u (cv) Distributivity: (u + v) w = u w + v w Zero Product: u 0 = 0 Jamie Monogan (UGA) Matrix & Linear Algebra 9 / 84

Matricies Matrix: A matrix is an array of mn real numbers arranged in m rows by n columns. a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn Jamie Monogan (UGA) Matrix & Linear Algebra 10 / 84

Matricies Note: vectors are special cases of matrices; a column vector of length k is a k 1 matrix; a row vector of length k is a 1 k matrix. Think of larger matrices as made up of a collection of row or column vectors. For example, A = ( a 1 a 2 a n ) b 1 B = b 2 b n Jamie Monogan (UGA) Matrix & Linear Algebra 11 / 84

Special Matricies 1 0 0 0 0 1 0 0 Identity: I n = 0 0.... 0 0... 1 1 1 1 1 1 1 1 1 J n = 1 1.... 1 1... 1 0 0 0 0 0 0 0 0 Zero: 0 = 0 0.... 0 0... 0 Jamie Monogan (UGA) Matrix & Linear Algebra 12 / 84

Special Matricies a 11 0 0 0 0 a 22 0 0 Diagonal:. 0 0... 0 0... a nn a 11 0 0 0 a 21 a 22 0 0 Lower Triangular:. a 31 a.. 32. a n1 a n2... a nn a 11 a 12... a 1n 0 a 22... a 2n Upper Triangular:. 0 0... 0 0... a nn Triangular: Either upper triangular or lower triangular Jamie Monogan (UGA) Matrix & Linear Algebra 13 / 84

Matrix Equality Let A and B be two m n matrices. Then A = B iff i {1,...,m}, j {1,...,n} a ij = b ij Jamie Monogan (UGA) Matrix & Linear Algebra 14 / 84

Matrix Addition Addition Let A and B be two m n matrices. Then a 11 + b 11 a 12 + b 12 a 1n + b 1n a 21 + b 21 a 22 + b 22 a 2n + b 2n A + B =...... a m1 + b m1 a m2 + b m2 a mn + b mn Note that matrices A and B must be the same size, in which case they are conformable for addition. Jamie Monogan (UGA) Matrix & Linear Algebra 15 / 84

Matrix Addition Example Addition Example: A = ( ) 1 2 3, B = 4 5 6 A + B = ( 1 2 ) 1 2 1 2 ( 2 4 ) 4 6 6 8 Jamie Monogan (UGA) Matrix & Linear Algebra 16 / 84

Scalar Multiplication Multiplication Scalar Multiplication: Given the scalar c, the scalar multiplication of ca is a 11 a 12 a 1n ca 11 ca 12 ca 1n a 21 a 22 a 2n ca = c...... = ca 21 ca 22 ca 2n...... a m1 a m2 a mn ca m1 ca m2 ca mn Jamie Monogan (UGA) Matrix & Linear Algebra 17 / 84

Scalar Multiplication Example Multiplication Example: ( ) 1 2 3 c = 2, A = 4 5 6 ( ) 2 4 6 ca = 8 10 12 Jamie Monogan (UGA) Matrix & Linear Algebra 18 / 84

Matrix Multiplication Multiplication Matrix Multiplication: If A is an m k matrix and B is a k n matrix, then their product C = AB is the m n matrix where c ij = a i1 b 1j + a i2 b 2j + + a ik b kj Consider A to be composed of stacked rows a i = ( ) a i1 a i2... a in, B to be composed of stacked columns b j = b 2j.... Then, AB = C, b mj where c ij = a i b j b 1j Jamie Monogan (UGA) Matrix & Linear Algebra 19 / 84

Matrix Multiplication Examples Multiplication 1 2 a b ( ) aa + bc ab + bd c d A B = ca + dc cb + dd C D e f ea + fc eb + fd ( ) 5 1 2 1 2 4 3 = 3 1 4 2 1 ( ) 1( 2) + 2(4) 1(2) 1(5) + 2( 3) 1(1) = 3( 2) + 1(4) + 4(2) 3(5) + 1( 3) + 4(1) ( ) 4 2 6 16 Jamie Monogan (UGA) Matrix & Linear Algebra 20 / 84

Multiplication Notes on Matrix Multiplication Note that the number of columns of the first matrix must equal the number of rows of the second matrix, in which case they are conformable for multiplication. The sizes of the matrices (including the resulting product) must be (m k)(k n) = (m n) Given AB, say B is pre-multiplied by A, B is left-multiplied by A, A is post-multiplied by B, A is right-multiplied by B Jamie Monogan (UGA) Matrix & Linear Algebra 21 / 84

Multiplication Properties of Matrix Algebra 1 Associative: (A + B) + C = A + (B + C) (AB)C = A(BC) 2 Commutative: A + B = B + A 3 Scalar Assoc & Comm: cab = (Ac)B = A(cB) = ABc 4 Scalar Distributive: c(b + C) = cb + cc (c + d)a = ca + da 5 Matrix Distributive: A(B + C) = AB + AC (A + B)C = AC + BC 6 Additive Identity: A + 0 = A 7 Multiplicative Identity: AI n = I m A = A 8 Zero Product: A 0 = 0 A = 0 Jamie Monogan (UGA) Matrix & Linear Algebra 22 / 84

Note of Warning: Multiplication Commutative law for multiplication does not hold the order of multiplication matters: AB BA In fact, AB may exist, while BA does not. Jamie Monogan (UGA) Matrix & Linear Algebra 23 / 84

Example Multiplication Example: ( ) 2 3 AB = 2 2 ( ) 1 7 BA = 1 3 A = ( ) 1 2, B = 1 3 ( ) 2 1 0 1 Jamie Monogan (UGA) Matrix & Linear Algebra 24 / 84

Transpose Transpose Transpose: The transpose of the m n matrix A is the n m matrix A T (or A ) obtained by interchanging the rows and columns of A. Examples: ( ) 1 4 2 3 A =, A 0 5 1 T = 4 0 2 5 3 1 2 B = 2 1, B T = ( 2 1 3 ) 3 Jamie Monogan (UGA) Matrix & Linear Algebra 25 / 84

Transpose Properties of Transpose 1 (A + B) T = A T + B T 2 (A T ) T = A 3 (ca) T = ca T 4 (AB) T = B T A T 5 (ABC... ) T = T...B T A T 6 A is symmetric iff A T = A 7 A is skew-symmetric iff A T = A Jamie Monogan (UGA) Matrix & Linear Algebra 26 / 84

Example Transpose Example of (AB) T = B T A T : A = ( ) 0 1 1 3 2, B = 2 2 2 1 3 3 1 (AB) T = ( ) 1 1 3 2 0 2 2 2 1 3 3 1 T ( ) 12 7 = 5 3 Jamie Monogan (UGA) Matrix & Linear Algebra 27 / 84

Example Continued Transpose B T A T = ( ) 2 0 2 3 1 3 1 = 1 2 1 2 3 ( ) 12 7 5 3 Jamie Monogan (UGA) Matrix & Linear Algebra 28 / 84

Transpose Square Matricies Square matrices have the same number of rows and columns; an n n square matrix is referred to as a matrix of order n. The diagonal of a square matrix is the vector of matrix elements that have the same subscripts. If A is a square matrix of order n, then its diagonal is [a 11,a 22,...,a nn ]. Jamie Monogan (UGA) Matrix & Linear Algebra 29 / 84

Types of Square Matricies Transpose Symmetric Matrix: A matrix A is symmetric if A = A ; this implies that a ij = a ji for all i and j. Examples: ( ) 1 2 A = = A 2 1 4 2 1, B = 2 1 3 = B 1 3 1 Jamie Monogan (UGA) Matrix & Linear Algebra 30 / 84

Types of Square Matricies Transpose Diagonal Matrix: A matrix A is diagonal if all of its non-diagonal entries are zero; formally, if a ij = 0 for all i j Examples: ( ) 1 0 A =, 0 2 4 0 0 B = 0 1 0 0 0 1 Jamie Monogan (UGA) Matrix & Linear Algebra 31 / 84

More Types of Square Matrix Transpose Triangular Matrix: A matrix is triangular one of two cases. If all entries below the diagonal are zero (a ij = 0 for all i > j), it is upper triangular. Conversely, if all entries above the diagonal are zero (a ij = 0 for all i < j), it is lower triangular. Examples: 1 0 0 1 7 4 A LT = 4 2 0, A UT = 0 3 9 3 2 5 0 0 3 Jamie Monogan (UGA) Matrix & Linear Algebra 32 / 84

More Types of Square Matrix Transpose Identity Matrix: The n n identity matrix I n is the matrix whose diagonal elements are 1 and all off-diagonal elements are 0. Examples: ( ) 1 0 0 1 0 I 2 =, I 0 1 3 = 0 1 0 0 0 1 Jamie Monogan (UGA) Matrix & Linear Algebra 33 / 84

Linear Equation Systems of Linear Equations Linear Equation: a 1 x 1 + a 2 x 2 + + a n x n = b a i are parameters or coefficients. x i are variables or unknowns. Linear : only one variable per term and degree at most 1. 1 R 2 : line x 2 = b a 2 a 1 a 2 x 1 2 R 3 : plane x 3 = a b 3 a 1 a 3 x 1 a 2 3 R n : hyperplane a 3 x 2 Jamie Monogan (UGA) Matrix & Linear Algebra 34 / 84

Systems of Linear Equations Systems of Linear Equations Often interested in solving linear systems like x 3y = 3 2x + y = 8 Jamie Monogan (UGA) Matrix & Linear Algebra 35 / 84

More Systems of Linear Equations Systems of Linear Equations More generally, we might have a system of m equations in n unknowns a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2... a m1 x 1 + a m2 x 2 + + a mn x n = b m This scalar system is equivalent to the matrix equation Ax = b A solution to a linear system of m equations in n unknowns is a set of n numbers x 1,x 2,,x n that satisfy each of the m equations. 1 R 2 : intersection of the lines. 2 R 3 : intersection of the planes. 3 R n : intersection of the hyperplanes. Jamie Monogan (UGA) Matrix & Linear Algebra 36 / 84

Example Systems of Linear Equations Example: x 3y = 3 2x + y = 8 x = 3 and y = 2 is the solution to the above 2 2 linear system. Graphically, the two lines intersect at (3,2). Jamie Monogan (UGA) Matrix & Linear Algebra 37 / 84

How many Solutions? Systems of Linear Equations Does a linear system have one, no, or multiple solutions? For a system of 2 equations in 2 unknowns (i.e., two lines): 1 One solution: The lines intersect at exactly one point. 2 No solution: The lines are parallel. 3 Infinite solutions: The lines coincide. Jamie Monogan (UGA) Matrix & Linear Algebra 38 / 84

Systems of Linear Equations Methods to Solve Linear Systems 1 Substitution 2 Elimination of variables 3 Matrix methods Jamie Monogan (UGA) Matrix & Linear Algebra 39 / 84

Systems of Linear Equations Method of Substitution Procedure: 1 Solve one equation for one variable, say x 1, in terms of the other variables in the equation. 2 Substitute the expression for x 1 into the other m 1 equations, resulting in a new system of m 1 equations in n 1 unknowns. 3 Repeat steps 1 and 2 until one equation in one unknown, say x n. We now have a value for x n. 4 Substitute x n into previous equation, a function of only x n. Repeat, using successive expressions of each variable in terms of the other variables, to find the values of all x i s. Jamie Monogan (UGA) Matrix & Linear Algebra 40 / 84

Exercise 1 Systems of Linear Equations Using substitution, solve: x 3y = 3 2x + y = 8 Jamie Monogan (UGA) Matrix & Linear Algebra 41 / 84

Exercise 2 Systems of Linear Equations Using substitution, solve x + 2y + 3z = 6 2x 3y + 2z = 14 3x + y z = 2 Jamie Monogan (UGA) Matrix & Linear Algebra 42 / 84

Elementary Equation Operations Systems of Linear Equations Elementary equation operations are used to transform the equations of a linear system, while maintaining an equivalent linear system equivalent in the sense that the same values of x j solve both the original and transformed systems. These operations are 1 Interchanging two equations, 2 Multiplying two sides of an equation by a constant, and 3 Adding equations to each other Jamie Monogan (UGA) Matrix & Linear Algebra 43 / 84

Interchanging Equations Systems of Linear Equations Interchanging Equations: Given the linear system a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 we can interchange its equations, resulting in the equivalent linear system a 21 x 1 + a 22 x 2 = b 2 a 11 x 1 + a 12 x 2 = b 1 Jamie Monogan (UGA) Matrix & Linear Algebra 44 / 84

Systems of Linear Equations Multiplying by a Constant Multiplying by a Constant: Suppose we had the following equation: 2 = 2 If we multiply each side of the equation by some number, say 4, we still have an equality: 2(4) = 2(4) = 8 = 8 More generally, we can multiply both sides of any equation by a constant and maintain an equivalent equation. For example, the following two equations are equivalent: a 11 x 1 + a 12 x 2 = b 1 ca 11 x 1 + ca 12 x 2 = cb 1 Jamie Monogan (UGA) Matrix & Linear Algebra 45 / 84

Systems of Linear Equations Adding Equations Adding Equations: Suppose we had the following two very simple equations: 3 = 3 7 = 7 If we add these two equations to each other, we get 7 + 3 = 7 + 3 = 10 = 10 Jamie Monogan (UGA) Matrix & Linear Algebra 46 / 84

Systems of Linear Equations More Adding Equations Suppose we now have a = b c = d If we add these two equations to each other, we get a + c = b + d Extending this, suppose we had the linear system a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 If we add these two equations to each other, we get (a 11 + a 21 )x 1 + (a 12 + a 22 )x 2 = b 1 + b 2 Jamie Monogan (UGA) Matrix & Linear Algebra 47 / 84

Method of Gaussian Elimination Systems of Linear Equations Gaussian Elimination is a method by which we: start with a linear system of m equations in n unknowns, use the elementary equation operations to eliminate variables, and arrive at an equivalent system of the form... Jamie Monogan (UGA) Matrix & Linear Algebra 48 / 84

Systems of Linear Equations Method of Gaussian Elimination Continued a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 22 x 2 + + a 2n x n = b 2.. a mnx n = b m Where a ij denotes the coefficient of the jth unknown in the ith equation after the above transformation. At each stage of the elimination process, we want to change some coefficient to 0 by adding a multiple of an earlier equation to the given equation. Jamie Monogan (UGA) Matrix & Linear Algebra 49 / 84

Systems of Linear Equations Method of Gaussian Elimination Continued The coefficients a 11, a 22, etc. are pivots. They are used to eliminate the variables in the rows below them in their respective columns. 1 Once the linear system is in the above reduced form, we then use back substitution to find the values of the x j s. 1 As we ll see, pivots don t need to be on the i = j diagonal. Additionally, sometimes when we pivot, we will eliminate variables in rows above a pivot. Jamie Monogan (UGA) Matrix & Linear Algebra 50 / 84

Example 1 Systems of Linear Equations Using Gaussian elimination, solve x 3y = 3 2x + y = 8 Jamie Monogan (UGA) Matrix & Linear Algebra 51 / 84

Example 2 Systems of Linear Equations Using Gaussian elimination, solve x + 2y + 3z = 6 2x 3y + 2z = 14 3x + y z = 2 Jamie Monogan (UGA) Matrix & Linear Algebra 52 / 84

Systems of Linear Equations Method of Gauss-Jordan Elimination The method of Gauss-Jordan elimination takes the Gaussian elimination method one step further. Once the linear system is in the reduced form shown in the preceding section, elementary row operations and Gaussian elimination are used to 1 Change the coefficient of the pivot term in each equation to 1 and 2 Eliminate all terms above each pivot in its column, resulting in a reduced, equivalent system. Jamie Monogan (UGA) Matrix & Linear Algebra 53 / 84

Systems of Linear Equations Gauss-Jordan Elimination Continued For a system of m equations in m unknowns, a typical reduced system would be x 1 = b 1 x 2 = b 2 x 3 =.... b 3 which needs no further work to solve for the x j s. x m = b m Jamie Monogan (UGA) Matrix & Linear Algebra 54 / 84

Example 1 Systems of Linear Equations Using Gauss-Jordan elimination, solve x 3y = 3 2x + y = 8 Jamie Monogan (UGA) Matrix & Linear Algebra 55 / 84

Example 2 Systems of Linear Equations Using Gauss-Jordan elimination, solve x + 2y + 3z = 6 2x 3y + 2z = 14 3x + y z = 2 Jamie Monogan (UGA) Matrix & Linear Algebra 56 / 84

Systems of Linear Equations Matricies to Represent Linear Systems are an efficient way to represent linear systems such as as Ax = b a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2... a m1 x 1 + a m2 x 2 + + a mn x n = b m Jamie Monogan (UGA) Matrix & Linear Algebra 57 / 84

Coefficient Matrix Systems of Linear Equations The m n coefficient matrix A is an array of mn real numbers arranged in m rows by n columns: a 11 a 12 a 1n a 21 a 22 a 2n A =..... a m1 a m2 a mn Jamie Monogan (UGA) Matrix & Linear Algebra 58 / 84

Variable & Output Vectors Systems of Linear Equations x 2 The unknown quantities are represented by the vector x =.. b 2 The RHS of the linear system is represented by the vector b =.. b m x 1 x n b 1 Jamie Monogan (UGA) Matrix & Linear Algebra 59 / 84

Augmented Matrix Systems of Linear Equations Augmented Matrix: When we append b to the coefficient matrix A, we get the augmented matrix  = [A b] a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2...... a m1 a m2 a mn b m Jamie Monogan (UGA) Matrix & Linear Algebra 60 / 84

Row Operations Systems of Linear Equations Elementary Row Operations: Elementary row operations, similar to elementary equation operations, transform some augmented matrix representation of a linear system into another augmented matrix that represents an equivalent linear system. Jamie Monogan (UGA) Matrix & Linear Algebra 61 / 84

Systems of Linear Equations Row Operations Since we re operating on equations when we operate on matrix rows, the elementary row operations correspond exactly to the equation operations: 1 (Interchange two rows) = (Interchange two equations) 2 (Multiply row by a constant) = (Multiply equation by a constant) 3 (Add two rows) = (Add two equations) Jamie Monogan (UGA) Matrix & Linear Algebra 62 / 84

Interchanging Rows Systems of Linear Equations Interchanging Rows: Suppose we have the augmented matrix ( ) a11 a  = 12 b 1 a 21 a 22 b 2 If we interchange the two rows, we get the augmented matrix ( a21 a 22 ) b 2 a 11 a 12 b 1 which represents linear system equivalent to that represented by Â. Jamie Monogan (UGA) Matrix & Linear Algebra 63 / 84

Multiplying by a Constant Systems of Linear Equations Multiplying by a Constant: If we multiply the second row of matrix  by a constant c, we get the augmented matrix ( ) a11 a 12 b 1 ca 21 ca 22 cb 2 which represents linear system equivalent to that represented by Â. Jamie Monogan (UGA) Matrix & Linear Algebra 64 / 84

Adding Rows Systems of Linear Equations Adding Rows: If we add the first row of matrix  to the second, we obtain the augmented matrix ( ) a11 a 12 b 1 a 11 + a 21 a 12 + a 22 b 1 + b 2 which represents linear system equivalent to that represented by Â. Jamie Monogan (UGA) Matrix & Linear Algebra 65 / 84

Elementary Matricies Systems of Linear Equations Elementary matrices come from performing an elem row op on I n When premultiplying A, elem matrices perform elem row ops on A Examples: ( ) 1 0 0 3 1 0 0 0 0 1 0 1 0 1 0 0 0 1 0 3 0 1 Jamie Monogan (UGA) Matrix & Linear Algebra 66 / 84

Elementary Matricies Example Systems of Linear Equations 1 0 0 1 0 2 3 Let E = 0 1 0 and A = 2 1 3 6. 3 0 1 1 4 4 0 Then 1 0 0 1 0 2 3 EA = 0 1 0 2 1 3 6 3 0 1 1 4 4 0 1 0 2 3 = 2 1 3 6 4 4 10 9 Jamie Monogan (UGA) Matrix & Linear Algebra 67 / 84

Row Echelon Form Systems of Linear Equations We use the row operations to change coefficients in the augmented matrix to 0 i.e., pivot to eliminate variables and to put it in a matrix form representing the final linear system of Gaussian elimination. An augmented matrix of the form a 11 a 12 a 13 a 1n b 1 0 a 22 a 23 a 2n b 2 0 0 a 33 a 3n b 3. 0 0 0.... 0 0 0 0 a mn b m is said to be in row echelon form each row has more leading zeros than the row preceding it. Jamie Monogan (UGA) Matrix & Linear Algebra 68 / 84

Reduced Row Echelon Form Systems of Linear Equations Reduced Row Echelon Form: Reduced row echelon form is the matrix representation of a linear system after Gauss-Jordan elimination. For a system of m equations in m unknowns, with no all-zero rows, the reduced row echelon form would be 1 0 0 0 0 b 1 0 1 0 0 0 b 2 0 0 1 0 0 b 3. 0 0 0.. 0. 0 0 0 0 1 b m Jamie Monogan (UGA) Matrix & Linear Algebra 69 / 84

Example 1 Systems of Linear Equations Using matrix methods, solve the following linear system by Gaussian elimination and then Gauss-Jordan elimination: x 3y = 3 2x + y = 8 Jamie Monogan (UGA) Matrix & Linear Algebra 70 / 84

Example 2 Systems of Linear Equations x + 2y + 3z = 6 2x 3y + 2z = 14 3x + y z = 2 Jamie Monogan (UGA) Matrix & Linear Algebra 71 / 84

Matrix Inverse Inverse of a Matrix An n n matrix A is nonsingular or invertible if there exists an n n matrix A 1 such that AA 1 = A 1 A = I n Then, A 1 is the inverse of A. If there is no such A 1, then A is singular or noninvertible. Jamie Monogan (UGA) Matrix & Linear Algebra 72 / 84

Example of Inverses Matrix Inverse Example: Let A = ( ) ( ) 2 3 1 3, B = 2 2 2 1 1 Since AB = BA = I n we conclude that B is the inverse of A, A 1, and that A is nonsingular. Jamie Monogan (UGA) Matrix & Linear Algebra 73 / 84

Properties of the Inverse Matrix Inverse Assume A,B, D nonsingular. Then, 1 A 1 exists. 2 A 1 is unique. 3 A 1 is nonsingular. 4 (A 1 ) 1 = A 5 (AB) 1 exists. 6 (AB) 1 = B 1 A 1 7 (A T ) 1 = (A 1 ) T 1 d 11 0 0 1 8 D 1 0 d = 22 0...... 1 0 0 d nn 9 I 1 = I 10 (ca) 1 = 1 c A 1 11 (A n ) 1 = A n Jamie Monogan (UGA) Matrix & Linear Algebra 74 / 84

Matrix Inverse Calculating the Inverse We know that if B is the inverse of A, then AB = BA = I n Looking only at the first and last parts of this AB = I n Solving for B is equivalent to solving for n linear systems, where each column of B is solved for the corresponding column in I n. Jamie Monogan (UGA) Matrix & Linear Algebra 75 / 84

Matrix Inverse Calculating the Inverse In performing Gauss-Jordan elimination for each individual system, the same row operations will be performed on A regardless of the column of B and I n. Hence, we can solve the systems simultaneously by augmenting A with I n and performing Gauss-Jordan elimination on A. Note that for the square matrix A, Gauss-Jordan elimination should result in A becoming row equivalent to I n. Therefore, if Gauss-Jordan elimination on [A I n ] results in [I n B], then B is the inverse of A. Otherwise, A is singular Jamie Monogan (UGA) Matrix & Linear Algebra 76 / 84

Calculating the Inverse Matrix Inverse 1 Form the augmented matrix [A I n ] 2 Using elementary row operations, transform the augmented matrix to reduced row echelon form. 3 The result of step 2 is an augmented matrix [C B]. 1 If C = I n, then B = A 1. 2 If C I n, then C has a row of zeros. A is singular and A 1 does not exist. Jamie Monogan (UGA) Matrix & Linear Algebra 77 / 84

Example Matrix Inverse 1 1 1 Find the inverse of A = 0 2 3. 5 5 1 ( ) 1 1 1 1 0 0 A I3 = 0 2 3 0 1 0 5 5 1 0 0 1 1 1 1 1 0 0 = 0 2 3 0 1 0 0 0 4 5 0 1 1 1 1 1 0 0 = 0 2 3 0 1 0 5 0 0 1 4 0 1 4 Jamie Monogan (UGA) Matrix & Linear Algebra 78 / 84

Example Matrix Inverse = = = = = 1 1 1 1 0 0 0 2 3 0 1 0 5 0 0 1 4 0 1 4 1 1 1 1 0 0 3 1 0 1 2 0 2 0 5 0 0 1 4 0 1 4 1 1 1 1 0 0 0 1 0 15 1 3 8 2 8 5 0 0 1 4 0 1 4 1 1 0 1 1 4 0 4 0 1 0 15 1 3 8 2 8 5 0 0 1 4 0 1 4 13 1 0 0 8 1 2 1 8 0 1 0 15 1 3 8 2 8 5 0 0 1 4 0 4 1 Jamie Monogan (UGA) Matrix & Linear Algebra 79 / 84

Check Matrix Inverse (See R check.) AA = = 13 1 1 1 8 2 1 1 8 0 2 3 1 3 8 2 8 5 5 1 5 4 0 4 1 1 0 0 0 1 0 0 0 1 A A = I 3 13 8 2 1 1 8 1 1 1 = 1 3 8 2 8 0 2 3 5 4 0 1 5 5 1 4 1 0 0 = 0 1 0 0 0 1 = I 3 A = A 1 Jamie Monogan (UGA) Matrix & Linear Algebra 80 / 84

Matrix Inverse Linear Systems and Inverses Let s return to the matrix representation of a linear system Ax = b If A is an n n matrix,then Ax = b is a system of n equations in n unknowns. Suppose A is nonsingular = A 1 exists. To solve this system, we can premultiply each side by A 1 and simplify: A 1 (Ax) = A 1 b (A 1 A)x = A 1 b I n x = A 1 b x = A 1 b Jamie Monogan (UGA) Matrix & Linear Algebra 81 / 84

Matrix Inverse Linear Systems and Inverses Hence, given A and b and given that A is nonsingular, then x = A 1 b is a unique solution to this system. Notice also that the requirements for A to be nonsingular correspond to the requirements for a linear system to have a unique solution: rank A = rows A = cols A. Jamie Monogan (UGA) Matrix & Linear Algebra 82 / 84

Linear Regression Parameters Matrix Inverse We often have linear systems with n obs and k indep vars. Let X be n k matrix of indep variable data, y be n 1 vector of dependent variable values, b be k 1 vector of linear parameters Then Xb = y Since usually n k, X not square To isolate b, how to make premultiplying matrix square? Observe: X (n k)(k n) X = (n n) can t premultiply b. But this can: X X = (k n)(n k) (k k) Jamie Monogan (UGA) Matrix & Linear Algebra 83 / 84

Matrix Inverse Linear Regression Parameters Xb = y X Xb = X y How to isolate b? Multiply by an inverse. (X X) 1 X Xb = (X X) 1 X y I k b = (X X) 1 X y b = (X X) 1 X y Befriend (X X) 1 X y. If you understand it, its cousins, and their properties (both strengths and weaknesses), your data-analytic future will be bright. Jamie Monogan (UGA) Matrix & Linear Algebra 84 / 84