Linear Equations and Matrix

Transcription

1 1/60 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra

2 Gaussian Elimination 2/60

3 Alpha Go Linear algebra begins with a system of linear equations. 3/60

4 Example { 1x + 2y = 3 4x + 5y = 6 elimination determinants x = y = 2, x = = 1, y = = 2 4/60

5 Perspectives row picture 1x + 2y = 3 4x + 5y = 6 solution: point of intersection column picture 1x + 2y = 3 4x + 5y = 6 solution: combination coefficients 5/60

6 Idea Eliminate unknowns in equations until it is easy to solve. recursion: reduce n unknowns in n equations to n 1 unknowns in n 1 equations base case: solve 1 unknown in 1 equation back substitution 6/60

7 Picture n equations in n unknowns (eli.) (sub.) (n 1) equations in (n 1) unknowns (eli.) (sub.). (eli.) (sub.) 1 equation in 1 unknown 7/60

8 Example 2u + v + w = 5 4u 6v = 2 2u + 7v + 2w = 9 2u + v + w = 5 8v 2w = 12 8v + 3w = 14 2u + v + w = 5 8v 2w = 12 w = 2 w = 2 v = 1 u = 1 8/60

9 Pivots and Multipliers unknown to eli. pivots multipliers u 2 2, -1 v -8-1 w 1-9/60

10 Complexity: Elimination Consider the elimination of the kth unknown using the kth equation. For each equation below 1 multiplication to determine the multiplier (n k + 2) multiplications to multiply the kth equation There are (n k) equations to eliminate the kth unknown from, so the total number of multiplications is (1 + n k + 2)(n k) There are (n 1) unknowns to eliminate, so the total number of multiplications is n 1 (1 + n k + 2)(n k) = O(n 3 ) k=1 10/60

11 Complexity: Back Substitution Consider solving the k th unknown counting from the last k 1 multiplications to substitute the values for the already-solved k 1 unknowns 1 multiplication to solve the k th unknown The total number of multiplications is k = k There are n unknowns to solve, so the total number of multiplications is n k = O(n 2 ) k =1 11/60

12 Matrix Multiplication 12/60

13 Definition Let a a 1l b b 1n A m l =....., B l n =..... a m1... a ml b l1... b ln The multiplication is a matrix with elements C = AB l c ij = a ik b kj k=1 of order m n. 13/60

14 Case: Dot Product For m = n = 1, we have AB = [ b ] 11 a a 1l. b l1 l = a 1k b k1 k=1 This is called dot product. 14/60

15 Case: Cross Product For l = 1, we have AB = a 11. a m1 This is called cross product. [ b11... b 1n ] a 11 b a 11 b 1n =..... a m1 b a m1 b 1n 15/60

16 Case: Linear Combination of Columns For n = 1 B = b 11. b l1 AB is a dot product if A is seen as a row of column vectors b 11 AB = a 1... a l. = a 1b a l b l1 b l1 l = a k b k1 k=1 It is a linear combination of the column vectors of A. 16/60

17 Case: Linear Combination of Rows For m = 1 A = [ a a 1l ] AB is a dot product if B is seen as a column of row vectors AB = [ ] a a 1l l = a 1k b k: k=1 b 1:. b l: = a 11b 1: + + a 1l b l: It is a linear combination of the row vectors of B. 17/60

18 Element by Element AB is a cross product if we treat A as a column of row vectors and B as a row of column vectors. AB = a 1:. a m: b 1... b n a 1: b 1... a 1: b n =..... a m: b 1... a m: b n AB is constructed one element at a time. 18/60

19 Matrix by Matrix AB is a dot product if we treat A as a row of column vectors and B as a column of row vectors. AB = a 1... a l = a 1 b 1: + + a l b l: l = a k b k: k=1 AB is constructed one matrix at a time. b 1:. b l: 19/60

20 Column by Column AB is a scalar product if we treat A as a whole and B as a row of column vectors. AB = A b 1... b n = Ab 1... Ab n AB is constructed one column vector at a time. The jth column vector Ab j = a k b kj k is a linear combination of the column vectors of A. 20/60

21 Row by Row AB is a scalar product if we treat A as a column of row vectors and B as a whole. AB = a 1:. a m: B = a 1: B. a m: B AB is constructed one row vector at a time. The ith row vector a i: B = a ik b k: k is a linear combination of the row vectors of B. 21/60

22 Example AB = ele. = mat. = col. = row = [ ] = [ 7 ] [ ] [ ] [ ] [ ] 1 [ 1 ] 2 [1 ] 3 [2 ] [ [ ] [ ] [ ] [ ] [ ] [ 1 1 ] + 2 [ 1 3 ] + 3 [ 2 0 ] 4 [ 1 1 ] + 2 [ 1 3 ] + 0 [ 2 0 ] [ ]] /60

23 Properties associative (AB)C = A(BC) distributive non-commutative A(B + C) = AB + AC (A + B)C = AC + BC AB BA 23/60

24 Example A = 2 1 0, B = 0 1 0, C = AB = BA? AC = CA? 24/60

25 Elimination and Matrix 25/60

26 A Linear Equation A linear equation a 1 x a n x n = b can be represented by a T x = b where a T = [ x ] 1 a 1... a n, x =. x n 26/60

27 A System of Linear Equations A system of linear equations can be represented by Ax = b where 2u + v + w = 5 4u 6v = 2 2u + 7v + 2w = 9 A = , x = Ax = b u 5 v, b = 2 w 9 27/60

28 Row Operation Subtract from a row by a multiple of another row The other rows are unchanged. a i: a i: ka j: 28/60

29 Elimination and Row Operation Elimination is equivalent to row operation. { 8v 2w = 12 8v + 3w = 14 { 8v 2w = 12 w = 2 is equivalent to [ 8 2 ] [ 8 2 ] /60

30 Gaussian Elimination Gaussian elimination is a sequence of row operations. Ax b { }} {{ }} { u v = w 9 Ux c { }} { {}} { u v = w 2 is equivalent to [A b] = = [U c] /60

31 Elementary Matrix An elementary matrix is almost the same as an identity matrix (remember?) except that there is one non-zero non-diagonal element I 3 = E 3 (2, 1; 2) = /60

32 Row Operation and Elementary Matrix Row operation is equivalent to multiplication of elementary matrix. a 1: a 2: a 3: row operation a 1: a 2: a 3: = a 1: a 2: 2a 1: a 3: a 1: a 2: 2a 1: a 3: row operation = /60

33 Elimination and Elementary Matrix Gaussian elimination is equivalent to a sequence of elementary matrices. [A b] = is equivalent to GFE [A b] = [U c] = [U c] where E = 2 1 0, F = 0 1 0, G = /60

34 Inverse of Row Operation The inverse of a row operation is a row operation. Consider a row operation that subtracts from the second row by twice the first row. To reverse this operation, one can adds to the second row by twice the first row E = E 1 = /60

35 Reversal where GFEA = U A = E 1 F 1 G 1 U E 1 F 1 G 1 = = = /60

36 LU Decomposition Barring row exchanges, a non-singular matrix A (meaning Ax = b has exactly one solution) can be factorized by A = LU where L is lower-triangular and U is upper-triangular = L and U are related to the multipliers and the pivots. 36/60

37 Permutation Matrix A permutation matrix is a re-arrangement of the rows or columns of an identity matrix P = /60

38 Multiplication by a Permutation Matrix From left P A {}} {{ }} { a 1: a 2: a 3: = a 1: a 3: a 2: From right A P { }} {{ }} { a 1 a 2 a = a 1 a 3 a /60

39 LU Decomposition with Permutation General LU decomposition of a non-singular matrix PA = LU where P is a permutation matrix. 39/60

40 Inverse 40/60

41 Definition Given A, there may be a matrix B such that AB = I = BA B is an inverse of A. 41/60

42 Uniqueness If an inverse of a matrix exists, then it is unique. The inverse of A is denoted by A 1 Let B 1 and B 2 be inverses of A. B 1 AB 2 = B 1 AB 2 (B 1 A)B 2 = B 1 (AB 2 ) B 2 = B 1 42/60

43 Relation to Singularity An invertible matrix is non-singular. Suppose A 1 exists. Ax = b A 1 Ax = A 1 b x = A 1 b 43/60

44 Inverse of Product (AB) 1 = B 1 A 1 (AB)(B 1 A 1 ) = ABB 1 A 1 = AA 1 = I 44/60

45 Computing Inverse Suppose A 1 exists, with A 1 = u v... z AA 1 = I A u v... z = i 1 i 2... i n Au = i 1, Av = i 2,..., Az = i n So A 1 can be found by Gaussian elimination. 45/60

46 Gauss-Jordan Method The columns of A 1 can be found simultaneously. [ A I ] row operations [ U L 1] row operations [ I U 1 L 1] = [ I A 1] 46/60

47 Example A = [ ] A I = = [ U L 1] = [ I U 1 L 1] = [ I A 1] /60

48 Gauss-Jordan with Row Exchange Let A be non-singular with PA = LU where P is a permutation matrix. Then [ A I ] row exchanges [ PA PI ] row operations [ I U 1 L 1 PI ] = [ I A 1 P 1 PI ] = [ I A 1] 48/60

49 Transpose 49/60

50 Definition The mirror image of a matrix across the diagonal line. [ ] T = /60

51 Transpose of a Product (AB) T = B T A T ( (AB) T ) ij = (AB) ji K = a jk b ki = k=1 K k=1 (B T ) ik (A T ) kj = ( B T A T ) ij 51/60

52 Transpose of an Inverse (A 1 ) T = (A T ) 1 AA 1 = I (AA 1 ) T = I (A 1 ) T (A T ) = I (A T ) 1 = (A 1 ) T 52/60

53 Symmetric Matrix A matrix is symmetric if A T = A R T R is symmetric (R T R) T = R T (R T ) T = (R T R) 53/60

54 Differential Equations and Linear Equations 54/60

55 Differential Equations in Physics classical mechanics electrodynamics (Maxwell equations) quantum mechanics (Schrödinger equation) 55/60

56 An Ordinary Differential Equation Consider an ordinary differential equation with boundary condition d 2 u(x) dx 2 = f (x), 0 x 1 u(0) = 0, u(1) = 0 56/60

57 A Computational Approach u(x) can be approximately and numerically solved. Consider discrete points x i = ih, i = 1, 2,..., n where Let h = ( 1 ) n + 1 u i = u(x i ) f i = f (x i ) 57/60

58 Approximation to the Derivatives du dx x=xi u(x i + x) u(x i ) = lim x 0 x u(x i+1) u(x i ) x i+1 x i = u i+1 u i h d 2 u dx 2 x=xi = d ( ) du dx dx x=xi du dx (x i) du (x dx i 1) h u i+1 2u i + u i 1 h 2 58/60

59 Substitution Through approximation is converted to d 2 u(x) dx 2 = f (x), 0 x 1 u i+1 + 2u i u i 1 = h 2 f i, i = 1,..., n 59/60

60 Example Consider n = 5, f (x) = 6x The system of linear equations for an approximate solution of the differential equation is u u u 3 = u u /60