Matrices and systems of linear equations

Transcription

1 Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T. Matrices Differential equations 1 / 61

2 Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 2 / 61

3 Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 3 / 61

4 Matrix Definition 1. A m n matrix is a rectangular array of numbers with m rows and n columns Example of 2 3 matrix: A = [ ] Samy T. Matrices Differential equations 4 / 61

5 Index notation Recall: we have [ 3 2 A = Index notation: For the matrix A we have ] a 12 = 2 3, a 21 = 0, a 23 = 3 7 Index notation for a m n matrix: a 11 a 12 a a 1n a A = 21 a 22 a a 2n a m1 a m2 a m3... a mn Samy T. Matrices Differential equations 5 / 61

6 Vectors A row 3-vector: a = [ ] 4 7 A column 5-vector: 1 4 b = π 67 3 Samy T. Matrices Differential equations 6 / 61

7 Transpose Definition 2. Let A = (a ij ) be a m n matrix Then A T is the matrix defined by a T ij = a ji Example: A = ] , A T = 4 7 [ Samy T. Matrices Differential equations 7 / 61

8 Square matrices Definition 3. Let A = (a ij ) be a m n matrix If m = n, then A is a square matrix Diagonal of a square matrix: Elements a ii. An example is A = Samy T. Matrices Differential equations 8 / 61

9 Square matrices (2) Diagonal matrix: Diag(1, 2, 3) = Symmetric matrix: such that A T = A. Example given by A = Skew-symmetric matrix: such that A T = A. Example given by A = Samy T. Matrices Differential equations 9 / 61

10 Matrix function Definition 4. A m n matrix function is a matrix whose elements are functions of a variable t. Example: A 2 3 matrix function A = [ t 2 cos(t) ] 3t 2 ln(t) e 5t t sin(t) Samy T. Matrices Differential equations 10 / 61

11 Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 11 / 61

12 Elementary operations on matrices Addition: A + B = (a ij ) + (b ij ) = (a ij + b ij ) Scalar multiplication: for α R αa = α(a ij ) = (αa ij ) Multiplication: If A is m n and B is n p, then n C = AB = c ij = a ik b kj k=1 Samy T. Matrices Differential equations 12 / 61

13 Rules for multiplications Identity: I n is a n n matrix defined by I n = Diag(1,..., 1) Rules to follow: A (B + C) = AB + AC A (BC) = (AB) C A 0 = 0 A = 0 A Id = Id A = A Distributive law Associative law Absorbing state Identity element Rule not to follow: AB BA in general. Samy T. Matrices Differential equations 13 / 61

14 Example Matrices: A = [ ] and B = [ ] Sum and scalar multiplication: A + B = [ ] 3 1, 2A = 1 1 [ ] Products: AB = [ ] and BA = [ ] Samy T. Matrices Differential equations 14 / 61

15 Dot product Definition 5. Let a and b two column n-vectors Then a b is the number defined by n a b = a T b = a k b k k=1 Example: If then 1 0 a = 3, b = a b = 7 Samy T. Matrices Differential equations 15 / 61

16 Properties of the transpose Theorem 6. Let A, C be two m n matrices B be a n p matrix Then 1 (A T ) T = A 2 (A + C) T = A T + C T 3 (AB) T = B T A T Samy T. Matrices Differential equations 16 / 61

17 Triangular matrices Example of upper triangular matrix: U = Example of lower triangular matrix: L = Samy T. Matrices Differential equations 17 / 61

18 Triangular matrices (2) Triangular matrices and products: 1 The product of two upper trg. mat. is an upper trg. mat. 2 The product of two lower trg. mat. is a lower trg. mat. Example: = Samy T. Matrices Differential equations 18 / 61

19 Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 19 / 61

20 Systems of linear equations General form of a m n linear system: a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2. (1) a m1 x 1 + a m2 x a mn x n = b m System coefficients: The real numbers a ij System constants: The real numbers b i Homogeneous system: When b i = 0 for all i Samy T. Matrices Differential equations 20 / 61

21 Example of linear system Linear system in R 3 : x 1 + x 2 + x 3 = 1 x 2 x 3 = 2 x 2 + x 3 = 6 Unique solution by substitution: x 1 = 5, x 2 = 4, x 3 = 2 Samy T. Matrices Differential equations 21 / 61

22 Notation R n Definition of R n : Set of ordered n-uples of real numbers (x 1,..., x n ) Matrix notation: An element of R n can be seen as a row or column n-vector x 1 x (x 1,..., x n ) 2 [x 1,..., x n ]. x n Samy T. Matrices Differential equations 22 / 61

23 Linear systems in R 3 General system in R 3 : a 11 x 1 + a 12 x 2 + a 13 x 3 = b 1 a 21 x 1 + a 22 x 2 + a 23 x 3 = b 2 a 31 x 1 + a 32 x 2 + a 33 x 3 = b 3 Geometrical interpretation: Intersection of 3 planes in R 3 Samy T. Matrices Differential equations 23 / 61

24 Sets of solutions Possible sets of solutions: In the general R n case we can have No solution to a linear system A unique solution An infinite number of solutions Definition 7. Consider a linear system given by (1). Then 1 If there is at least one solution the system is consistent 2 If there is no solution the system is inconsistent Samy T. Matrices Differential equations 24 / 61

25 Related matrices Matrix of coefficients: For the system (1), given by a 11 a 12 a a 1n a A = 21 a 22 a a 2n a m1 a m2 a m3... a mn Augmented matrix: For the system (1), given by a 11 a 12 a a 1n b 1 A a = 21 a 22 a a 2n b a m1 a m2 a m3... a mn b m Samy T. Matrices Differential equations 25 / 61

26 Vector formulation Example of system in R 3 : x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Related matrices: A = 2 5 1, b = 5, x = Vector formulation of the system: Ax = b x 1 x 2 x 3 Samy T. Matrices Differential equations 26 / 61

27 Systems of differential equations Example of system in R 2 x 1 = 3x 1 + sin(t)x 2 + e t x 2 = 7tx 1 + t 2 x 2 4e t Related matrices: [ ] [ ] 3 sin(t) e t A(t) = 7t t 2, b(t) = 4e t, x(t) = Vector formulation of the system: x (t) = A(t) x(t) + b(t) [ ] x1 (t) x 2 (t) Samy T. Matrices Differential equations 27 / 61

28 Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 28 / 61

29 Elementary row operations Operations leaving the system unchanged: 1 Permute equations 2 Multiply a row by a nonzero constant 3 Add a multiple of one equation to another equation Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 29 / 61

30 Example of elementary row operations Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Permute R 1 and R 2 : Denoted by P 12 2x 1 +5x 2 x 3 = 5 x 1 +3x 2 4x 3 = 1 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 30 / 61

31 Example of elementary row operations (2) Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Multiply R 2 by 2: Denoted by M 2 ( 2) 2x 1 +5x 2 x 3 = 5 4x 1 10x 2 +2x 3 = 10 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 31 / 61

32 Example of elementary row operations (3) Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Add 2R 3 to R 1 : Denoted by A 31 (2) 3x 1 +3x 2 +8x 3 = 7 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Samy T. Matrices Differential equations 32 / 61

33 Elementary operations in matrix form Example of system: x 1 +3x 2 4x 3 = 1 2x 1 +5x 2 x 3 = 5 x 1 +6x 3 = 3 Adding 2R 3 to R 1 : A 31 (2) Samy T. Matrices Differential equations 33 / 61

34 Row-echelon matrices Definition 8. A m n matrix is row-echelon whenever 1 If there are rows consisting only of 0 s They are at the bottom of the matrix 2 The first nonzero element in a nonzero row is a 1 Called leading 1 3 The leading 1 s have a triangular shape Samy T. Matrices Differential equations 34 / 61

35 Example of row-echelon matrix Row-echelon matrix: Related system: x 1 +x 2 x 3 = 4 +x 2 3x 3 = 5 x 3 = 2 Solution of the system: very easy thanks to a back substitution x 3 = 2, x 2 = 11, x 1 = 5 Strategy to solve general systems: Reduction to a row-echelon system Samy T. Matrices Differential equations 35 / 61

36 Reduction to a row-echelon system Algorithm: 1 Start with a m n matrix A. If A = 0, go to step 7. 2 Pivot column: leftmost nonzero column. Pivot position: topmost position in the pivot column. 3 Use elementary row operations to put 1 in the pivot position. 4 Use elementary row operations to put zeros below pivot position. 5 If all rows below pivot are 0, go to step 7. 6 Otherwise apply steps 2 to 5 to the rows below pivot position. 7 The matrix is row-echelon. Samy T. Matrices Differential equations 36 / 61

37 Example of reduction First operations: P A 12( 3), A 13 ( 1) Iteration: M 2 ( 1) A 23(1) M 3(1/4) Samy T. Matrices Differential equations 37 / 61

38 Rank of a matrix Definition 9. The number of nonzero rows in a row-echelon form of A is called rank(a). Example: If A = we have seen that a row-echelon form of A is Therefore rank(a) = 3. Samy T. Matrices Differential equations 38 / 61

39 Reduced row-echelon matrices Definition 10. A reduced row-echelon matrix is a matrix A such that 1 A is row-echelon 2 Any column with a leading 1 has zeros everywhere else Samy T. Matrices Differential equations 39 / 61

40 Example of reduced row-echelon Example: M 2(1/14) A 21( 9), A 23 (14) In particular rank(a) = 2 Samy T. Matrices Differential equations 40 / 61

41 Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 41 / 61

42 Basic idea of Gaussian elimination Aim: Solve a linear system of equations Strategy: 1 Reduce the augmented matrix to row-echelon 2 Solve the system backward thanks to the row-echelon form Samy T. Matrices Differential equations 42 / 61

43 Example of Gaussian elimination System: Augmented matrix: 3x 1 2x 2 +2x 3 = 9 x 1 2x 2 +x 3 = 5 2x 1 x 2 2x 3 = A = Samy T. Matrices Differential equations 43 / 61

44 Example of Gaussian elimination (2) Row-echelon form of the augmented matrix: A System corresponding to the augmented matrix: x 1 2x 2 +x 3 = 5 x 2 +3x 3 = 5 x 3 = 2 Solution set: S = {(1, 1, 2)} Samy T. Matrices Differential equations 44 / 61

45 Example of Gauss-Jordan elimination System: Augmented matrix: x 1 +2x 2 x 3 = 1 2x 1 +5x 2 x 3 = 3 x 1 +3x 2 +2x 3 = A = Samy T. Matrices Differential equations 45 / 61

46 Example of Gauss-Jordan elimination (2) Reduced row-echelon form of the augmented matrix: A System corresponding to the augmented matrix: x 1 = 5 x 2 = 1 x 3 = 2 Solution set: S = {(5, 1, 2)} Samy T. Matrices Differential equations 46 / 61

47 Comparison between Gauss and Gauss-Jordan Pros and cons: For Gauss-Jordan, the final backward system is easier to solve The reduced row-echelon form is costly in terms of computations Overall, Gauss is more efficient than Gauss-Jordan for systems Main interest of Gauss-Jordan: One can compute the inverse of a matrix Samy T. Matrices Differential equations 47 / 61

48 Consistency of a system Theorem 11. Consider Then A m n system of the form Ax = b r = rank(a) and r = rank(a ) 1 If r < r, the system is inconsistent. 2 If r = r, the system is consistent and (a) There is a unique solution iff r = n. (b) There is an infinite number of solutions iff r < n. Samy T. Matrices Differential equations 48 / 61

49 Ex. of system with number of solutions System: x 1 2x 2 +2x 3 x 4 = 3 3x 1 +x 2 +6x 3 +11x 4 = 16 2x 1 x 2 +4x 3 +4x 4 = 9 (2) Row-echelon form of the augmented matrix: A Consistency: We have r = r = 2 Consistent system with infinite number of solutions Samy T. Matrices Differential equations 49 / 61

50 Ex. of system with number of solutions (2) Rule for systems with number of solutions: Choose as free variables those variables that do not correspond to a leading 1 in row-echelon form of A Application to system (2): Free variables: x 3 = s and x 4 = t Solution set: S = {(5 2s 3t, 1 2t, s, t); s, t R} = {(5, 1, 0, 0) + s ( 2, 0, 1, 0) + t ( 3, 2, 0, 1); s, t R} Samy T. Matrices Differential equations 50 / 61

51 Homogeneous systems Proposition 12. For a m n matrix A, consider the system Then we have A x = 0. 1 The system is always consistent with trivial solution x = 0. 2 If m < n, the system has number of solutions. 3 If m n, then (a) If r < n, the system has number of solutions. (b) If r = n, the solution x = 0 is unique. Samy T. Matrices Differential equations 51 / 61

52 Example of homogeneous system Matrix: System: Augmented matrix: A = A x = A = Samy T. Matrices Differential equations 52 / 61

53 Example of homogeneous system (2) Reduced echelon form of A : Number of solutions: We have m = n = 3 and r = 2 < n infinite number of solutions Set of solutions: S = {(s, 0, 0); s R}. Samy T. Matrices Differential equations 53 / 61

54 Outline 1 Matrices: definitions and notation 2 Matrix algebra 3 Terminology for systems of linear equations 4 Row operations 5 Gaussian elimination 6 The inverse of a square matrix Samy T. Matrices Differential equations 54 / 61

55 Definition of inverse Problem: Consider a n n matrix A. We wish to find B such that A B = I n, and B A = I n (3) Definition 13. If B satisfies (3), we set B = A 1. A 1 is called the inverse of A. Remark on notation: A 1 does not mean 1 A has no meaning unless n = 1, i.e A R 1 A Samy T. Matrices Differential equations 55 / 61

56 Relation with previous notions Inverse and systems: If A is invertible, then A x = b x = A 1 b Inverse and rank: A is invertible iff rank(a) = n Samy T. Matrices Differential equations 56 / 61

57 Gauss-Jordan technique (for 3 3 matrices) Method: 1 Form an augmented matrix of the form a 11 a 12 a A = a 21 a 22 a a 31 a 32 a Use the Gauss-Jordan reduction technique, which yields 3 Then B = A 1 A b 11 b 12 b b 21 b 22 b b 31 b 32 b 33 B Samy T. Matrices Differential equations 57 / 61

58 2-d example Matrix: Augmented matrix: A = A = [ ] [ ] Gauss-Jordan reduced form: A [ ] Inverse: A 1 = [ ] Samy T. Matrices Differential equations 58 / 61

59 2-d example (2) System: Related matrices: Solution: Solution set: A = x = A 1 b = x 1 +2x 2 = 3 x 1 +x 2 = 2 [ ] 1 2, b = 1 1 [ ] 3 2 [ ] [ ] S = {(1, 1)} = [ ] 1 1 Samy T. Matrices Differential equations 59 / 61

60 Properties of the inverse Proposition 14. Let A and B invertible n n matrices Then 1 (A 1 ) 1 = A 2 (AB) 1 = B 1 A 1 3 (A T ) 1 = (A 1 ) T Samy T. Matrices Differential equations 60 / 61

61 Checking property 2 Example of matrices: A = [ ] 1 2, B = 1 1 [ ] AB = [ ] Inverses: Rule #2: A 1 = [ ] 1 2, B 1 = 1 1 (AB) 1 = B 1 A 1 = [ 1 ] [ ] Samy T. Matrices Differential equations 61 / 61