Matrices and systems of linear equations

Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T. Matrices Differential equations 1 / 61

Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 2 / 61

Matrix Definition 1. A m n matrix is a rectangular array of numbers with m rows and n columns Example of 2 3 matrix: A = [ 3 2 2 3 1 5 0 5 4 3 7 ] Samy T. Matrices Differential equations 4 / 61

Index notation Recall: we have [ 3 2 A = 0 5 3 4 7 Index notation: For the matrix A we have 2 3 1 5 ] a 12 = 2 3, a 21 = 0, a 23 = 3 7 Index notation for a m n matrix: a 11 a 12 a 13... a 1n a A = 21 a 22 a 23... a 2n....... a m1 a m2 a m3... a mn Samy T. Matrices Differential equations 5 / 61

Vectors A row 3-vector: a = [ 2 3 1 5 ] 4 7 A column 5-vector: 1 4 b = π 67 3 Samy T. Matrices Differential equations 6 / 61

Transpose Definition 2. Let A = (a ij ) be a m n matrix Then A T is the matrix defined by a T ij = a ji Example: A = ] 2 1 3 5 0 5 3, A T = 4 7 [ 3 2 3 0 2 2 5 3 4 1 3 5 7 Samy T. Matrices Differential equations 7 / 61

Square matrices Definition 3. Let A = (a ij ) be a m n matrix If m = n, then A is a square matrix Diagonal of a square matrix: Elements a ii. An example is 1 7 4 A = 2 9 0 8 5 5 Samy T. Matrices Differential equations 8 / 61

Square matrices (2) Diagonal matrix: 1 0 0 Diag(1, 2, 3) = 0 2 0 0 0 3 Symmetric matrix: such that A T = A. Example given by 1 7 4 A = 7 9 0 4 0 5 Skew-symmetric matrix: such that A T = A. Example given by 0 7 4 A = 7 0 0 4 0 0 Samy T. Matrices Differential equations 9 / 61

Matrix function Definition 4. A m n matrix function is a matrix whose elements are functions of a variable t. Example: A 2 3 matrix function A = [ t 2 cos(t) ] 3t 2 ln(t) e 5t t sin(t) Samy T. Matrices Differential equations 10 / 61

Elementary operations on matrices Addition: A + B = (a ij ) + (b ij ) = (a ij + b ij ) Scalar multiplication: for α R αa = α(a ij ) = (αa ij ) Multiplication: If A is m n and B is n p, then n C = AB = c ij = a ik b kj k=1 Samy T. Matrices Differential equations 12 / 61

Rules for multiplications Identity: I n is a n n matrix defined by I n = Diag(1,..., 1) Rules to follow: A (B + C) = AB + AC A (BC) = (AB) C A 0 = 0 A = 0 A Id = Id A = A Distributive law Associative law Absorbing state Identity element Rule not to follow: AB BA in general. Samy T. Matrices Differential equations 13 / 61

Example Matrices: A = [ 1 2 0 2 ] and B = [ 2 1 1 1 ] Sum and scalar multiplication: A + B = [ ] 3 1, 2A = 1 1 [ ] 2 4 0 4 Products: AB = [ 0 3 2 2 ] and BA = [ 2 2 1 4 ] Samy T. Matrices Differential equations 14 / 61

Dot product Definition 5. Let a and b two column n-vectors Then a b is the number defined by n a b = a T b = a k b k k=1 Example: If then 1 0 a = 3, b = 2 1 1 a b = 7 Samy T. Matrices Differential equations 15 / 61

Properties of the transpose Theorem 6. Let A, C be two m n matrices B be a n p matrix Then 1 (A T ) T = A 2 (A + C) T = A T + C T 3 (AB) T = B T A T Samy T. Matrices Differential equations 16 / 61

Triangular matrices Example of upper triangular matrix: 1 7 4 U = 0 9 0 0 0 5 Example of lower triangular matrix: 1 0 0 L = 7 9 0 4 0 5 Samy T. Matrices Differential equations 17 / 61

Triangular matrices (2) Triangular matrices and products: 1 The product of two upper trg. mat. is an upper trg. mat. 2 The product of two lower trg. mat. is a lower trg. mat. Example: 1 7 4 1 1 4 1 62 16 0 9 0 0 9 0 = 0 81 0 0 0 5 0 0 5 0 0 25 Samy T. Matrices Differential equations 18 / 61

Systems of linear equations General form of a m n linear system: a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2. (1) a m1 x 1 + a m2 x 2 + + a mn x n = b m System coefficients: The real numbers a ij System constants: The real numbers b i Homogeneous system: When b i = 0 for all i Samy T. Matrices Differential equations 20 / 61

Example of linear system Linear system in R 3 : x 1 + x 2 + x 3 = 1 x 2 x 3 = 2 x 2 + x 3 = 6 Unique solution by substitution: x 1 = 5, x 2 = 4, x 3 = 2 Samy T. Matrices Differential equations 21 / 61

Notation R n Definition of R n : Set of ordered n-uples of real numbers (x 1,..., x n ) Matrix notation: An element of R n can be seen as a row or column n-vector x 1 x (x 1,..., x n ) 2 [x 1,..., x n ]. x n Samy T. Matrices Differential equations 22 / 61

Linear systems in R 3 General system in R 3 : a 11 x 1 + a 12 x 2 + a 13 x 3 = b 1 a 21 x 1 + a 22 x 2 + a 23 x 3 = b 2 a 31 x 1 + a 32 x 2 + a 33 x 3 = b 3 Geometrical interpretation: Intersection of 3 planes in R 3 Samy T. Matrices Differential equations 23 / 61

Sets of solutions Possible sets of solutions: In the general R n case we can have No solution to a linear system A unique solution An infinite number of solutions Definition 7. Consider a linear system given by (1). Then 1 If there is at least one solution the system is consistent 2 If there is no solution the system is inconsistent Samy T. Matrices Differential equations 24 / 61

Related matrices Matrix of coefficients: For the system (1), given by a 11 a 12 a 13... a 1n a A = 21 a 22 a 23... a 2n....... a m1 a m2 a m3... a mn Augmented matrix: For the system (1), given by a 11 a 12 a 13... a 1n b 1 A a = 21 a 22 a 23... a 2n b 2........ a m1 a m2 a m3... a mn b m Samy T. Matrices Differential equations 25 / 61

Vector formulation Example of system in R 3 : x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Related matrices: 1 3 4 1 A = 2 5 1, b = 5, x = 1 0 6 3 Vector formulation of the system: Ax = b x 1 x 2 x 3 Samy T. Matrices Differential equations 26 / 61

Systems of differential equations Example of system in R 2 x 1 = 3x 1 + sin(t)x 2 + e t x 2 = 7tx 1 + t 2 x 2 4e t Related matrices: [ ] [ ] 3 sin(t) e t A(t) = 7t t 2, b(t) = 4e t, x(t) = Vector formulation of the system: x (t) = A(t) x(t) + b(t) [ ] x1 (t) x 2 (t) Samy T. Matrices Differential equations 27 / 61

Elementary row operations Operations leaving the system unchanged: 1 Permute equations 2 Multiply a row by a nonzero constant 3 Add a multiple of one equation to another equation Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 29 / 61

Example of elementary row operations Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Permute R 1 and R 2 : Denoted by P 12 2x 1 +5x 2 x 3 = 5 x 1 +3x 2 4x 3 = 1 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 30 / 61

Example of elementary row operations (2) Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Multiply R 2 by 2: Denoted by M 2 ( 2) 2x 1 +5x 2 x 3 = 5 4x 1 10x 2 +2x 3 = 10 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 31 / 61

Example of elementary row operations (3) Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Add 2R 3 to R 1 : Denoted by A 31 (2) 3x 1 +3x 2 +8x 3 = 7 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 32 / 61

Elementary operations in matrix form Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Adding 2R 3 to R 1 : 1 3 4 1 2 5 1 5 1 0 6 3 A 31 (2) 3 3 8 7 2 5 1 5 1 0 6 3 Samy T. Matrices Differential equations 33 / 61

Row-echelon matrices Definition 8. A m n matrix is row-echelon whenever 1 If there are rows consisting only of 0 s They are at the bottom of the matrix 2 The first nonzero element in a nonzero row is a 1 Called leading 1 3 The leading 1 s have a triangular shape Samy T. Matrices Differential equations 34 / 61

Example of row-echelon matrix Row-echelon matrix: 1 1 1 4 0 1 3 5 0 0 1 2 Related system: x 1 +x 2 x 3 = 4 +x 2 3x 3 = 5 x 3 = 2 Solution of the system: very easy thanks to a back substitution x 3 = 2, x 2 = 11, x 1 = 5 Strategy to solve general systems: Reduction to a row-echelon system Samy T. Matrices Differential equations 35 / 61

Reduction to a row-echelon system Algorithm: 1 Start with a m n matrix A. If A = 0, go to step 7. 2 Pivot column: leftmost nonzero column. Pivot position: topmost position in the pivot column. 3 Use elementary row operations to put 1 in the pivot position. 4 Use elementary row operations to put zeros below pivot position. 5 If all rows below pivot are 0, go to step 7. 6 Otherwise apply steps 2 to 5 to the rows below pivot position. 7 The matrix is row-echelon. Samy T. Matrices Differential equations 36 / 61

Example of reduction First operations: 3 2 5 2 1 1 1 1 1 1 1 1 1 1 1 1 P 12 3 2 5 2 A 12( 3), A 13 ( 1) 0 1 2 1 1 0 3 4 1 0 3 4 0 1 2 3 Iteration: 1 1 1 1 1 1 1 1 1 1 1 1 M 2 ( 1) 0 1 2 1 A 23(1) 0 1 2 1 M 3(1/4) 0 1 2 1 0 1 2 3 0 0 0 4 0 0 0 1 Samy T. Matrices Differential equations 37 / 61

Rank of a matrix Definition 9. The number of nonzero rows in a row-echelon form of A is called rank(a). Example: If 3 2 5 2 A = 1 1 1 1 1 0 3 4 we have seen that a row-echelon form of A is 1 1 1 1 0 1 2 1 0 0 0 1 Therefore rank(a) = 3. Samy T. Matrices Differential equations 38 / 61

Reduced row-echelon matrices Definition 10. A reduced row-echelon matrix is a matrix A such that 1 A is row-echelon 2 Any column with a leading 1 has zeros everywhere else Samy T. Matrices Differential equations 39 / 61

Example of reduced row-echelon Example: 1 9 26 1 9 26 1 0 8 0 14 28 M 2(1/14) 0 1 2 A 21( 9), A 23 (14) 0 1 2 0 14 28 0 14 28 0 0 0 In particular rank(a) = 2 Samy T. Matrices Differential equations 40 / 61

Basic idea of Gaussian elimination Aim: Solve a linear system of equations Strategy: 1 Reduce the augmented matrix to row-echelon 2 Solve the system backward thanks to the row-echelon form Samy T. Matrices Differential equations 42 / 61

Example of Gaussian elimination System: Augmented matrix: 3x 1 2x 2 +2x 3 = 9 x 1 2x 2 +x 3 = 5 2x 1 x 2 2x 3 = 1 3 2 2 9 A = 1 2 1 5 2 1 2 1 Samy T. Matrices Differential equations 43 / 61

Example of Gaussian elimination (2) Row-echelon form of the augmented matrix: 1 2 1 5 A 0 1 3 5 0 0 1 2 System corresponding to the augmented matrix: x 1 2x 2 +x 3 = 5 x 2 +3x 3 = 5 x 3 = 2 Solution set: S = {(1, 1, 2)} Samy T. Matrices Differential equations 44 / 61

Example of Gauss-Jordan elimination System: Augmented matrix: x 1 +2x 2 x 3 = 1 2x 1 +5x 2 x 3 = 3 x 1 +3x 2 +2x 3 = 6 1 2 1 1 A = 2 5 1 3 1 3 2 6 Samy T. Matrices Differential equations 45 / 61

Example of Gauss-Jordan elimination (2) Reduced row-echelon form of the augmented matrix: 1 0 0 5 A 0 1 0 1 0 0 1 2 System corresponding to the augmented matrix: x 1 = 5 x 2 = 1 x 3 = 2 Solution set: S = {(5, 1, 2)} Samy T. Matrices Differential equations 46 / 61

Comparison between Gauss and Gauss-Jordan Pros and cons: For Gauss-Jordan, the final backward system is easier to solve The reduced row-echelon form is costly in terms of computations Overall, Gauss is more efficient than Gauss-Jordan for systems Main interest of Gauss-Jordan: One can compute the inverse of a matrix Samy T. Matrices Differential equations 47 / 61

Consistency of a system Theorem 11. Consider Then A m n system of the form Ax = b r = rank(a) and r = rank(a ) 1 If r < r, the system is inconsistent. 2 If r = r, the system is consistent and (a) There is a unique solution iff r = n. (b) There is an infinite number of solutions iff r < n. Samy T. Matrices Differential equations 48 / 61

Ex. of system with number of solutions System: x 1 2x 2 +2x 3 x 4 = 3 3x 1 +x 2 +6x 3 +11x 4 = 16 2x 1 x 2 +4x 3 +4x 4 = 9 (2) Row-echelon form of the augmented matrix: 1 2 2 1 3 A 0 1 0 2 1 0 0 0 0 0 Consistency: We have r = r = 2 Consistent system with infinite number of solutions Samy T. Matrices Differential equations 49 / 61

Ex. of system with number of solutions (2) Rule for systems with number of solutions: Choose as free variables those variables that do not correspond to a leading 1 in row-echelon form of A Application to system (2): Free variables: x 3 = s and x 4 = t Solution set: S = {(5 2s 3t, 1 2t, s, t); s, t R} = {(5, 1, 0, 0) + s ( 2, 0, 1, 0) + t ( 3, 2, 0, 1); s, t R} Samy T. Matrices Differential equations 50 / 61

Homogeneous systems Proposition 12. For a m n matrix A, consider the system Then we have A x = 0. 1 The system is always consistent with trivial solution x = 0. 2 If m < n, the system has number of solutions. 3 If m n, then (a) If r < n, the system has number of solutions. (b) If r = n, the solution x = 0 is unique. Samy T. Matrices Differential equations 51 / 61

Example of homogeneous system Matrix: System: Augmented matrix: 0 2 3 A = 0 1 1 0 3 7 A x = 0. 0 2 3 0 A = 0 1 1 0 0 3 7 0 Samy T. Matrices Differential equations 52 / 61

Example of homogeneous system (2) Reduced echelon form of A : 0 1 0 0 0 0 1 0 0 0 0 0 Number of solutions: We have m = n = 3 and r = 2 < n infinite number of solutions Set of solutions: S = {(s, 0, 0); s R}. Samy T. Matrices Differential equations 53 / 61

Definition of inverse Problem: Consider a n n matrix A. We wish to find B such that A B = I n, and B A = I n (3) Definition 13. If B satisfies (3), we set B = A 1. A 1 is called the inverse of A. Remark on notation: A 1 does not mean 1 A has no meaning unless n = 1, i.e A R 1 A Samy T. Matrices Differential equations 55 / 61

Relation with previous notions Inverse and systems: If A is invertible, then A x = b x = A 1 b Inverse and rank: A is invertible iff rank(a) = n Samy T. Matrices Differential equations 56 / 61

Gauss-Jordan technique (for 3 3 matrices) Method: 1 Form an augmented matrix of the form a 11 a 12 a 13 1 0 0 A = a 21 a 22 a 23 0 1 0 a 31 a 32 a 33 0 0 1 2 Use the Gauss-Jordan reduction technique, which yields 3 Then B = A 1 A 1 0 0 b 11 b 12 b 13 0 1 0 b 21 b 22 b 23 0 0 1 b 31 b 32 b 33 B Samy T. Matrices Differential equations 57 / 61

2-d example Matrix: Augmented matrix: A = A = [ ] 1 2 1 1 [ 1 2 1 ] 0 1 1 0 1 Gauss-Jordan reduced form: A [ 1 0 1 ] 2 0 1 1 1 Inverse: A 1 = [ ] 1 2 1 1 Samy T. Matrices Differential equations 58 / 61

2-d example (2) System: Related matrices: Solution: Solution set: A = x = A 1 b = x 1 +2x 2 = 3 x 1 +x 2 = 2 [ ] 1 2, b = 1 1 [ ] 3 2 [ ] [ ] 1 2 3 1 1 2 S = {(1, 1)} = [ ] 1 1 Samy T. Matrices Differential equations 59 / 61

Properties of the inverse Proposition 14. Let A and B invertible n n matrices Then 1 (A 1 ) 1 = A 2 (AB) 1 = B 1 A 1 3 (A T ) 1 = (A 1 ) T Samy T. Matrices Differential equations 60 / 61

Checking property 2 Example of matrices: A = [ ] 1 2, B = 1 1 [ ] 3 1 2 1 AB = [ ] 7 3 5 2 Inverses: Rule #2: A 1 = [ ] 1 2, B 1 = 1 1 (AB) 1 = B 1 A 1 = [ 1 ] 1 2 3 [ ] 2 3 5 7 Samy T. Matrices Differential equations 61 / 61