10.34: Numerical Methods Applied to Chemical Engineering. Lecture 2: More basics of linear algebra Matrix norms, Condition number

Transcription

1 10.34: Numerical Methods Applied to Chemical Engineering Lecture 2: More basics of linear algebra Matrix norms, Condition number 1

2 Recap Numerical error Operations Properties Scalars, vectors, and matrices 2

3 Recap Vectors: What mathematical object is the equivalent of an infinite dimensional vector? 3

4 Vectors: What mathematical object is the equivalent of an infinite dimensional vector? A function. 4

5 0 A11 A A 1M 1 Ordered sets of numbers: B A 21 A A 2M A = C A A N1 A N2... A NM Set of all real matrices with N rows and M columns, R N M Addition: C = A + B ) C ij = A ij + B ij Multiplication by scalar: C = ca ) C ij = ca ij Transpose: C = AT ) C ij = A ji Trace (square matrices): N Tr A = X A ii i=1 5

6 M Matrix-vector product: y = Ax ) y i = X A ij x j j=1 M Matrix-matrix product: C = AB ) C ij = X A ik B kj Properties: k=1 no commutation in general: AB = 6 BA association: A(BC) =(AB)C distribution: A(B + C) =AB + AC transposition: (AB)T = B T A T inversion: A -1 A = AA -1 = I if det(a) 6=0 6

7 Matrix-matrix product: Vectors are matrices too: x 2 RN x 2 R N 1 T y 2 R N y T 2 R 1 N T What is: y x? M C = AB ) C ij = X A ik B kj k=1 7

8 Matrix-matrix product: Vectors are matrices too: x 2 RN x 2 R N 1 y T 2 RN y T 2 R 1 N T What is: y x? A scalar: y T x = y x 8

9 Examples: A, B 2 RN N x 2 R N How many operations to compute: Ax AB ABx What is x T ABx? What is ABxx T? 9

10 Dyadic product: A = xy T = x y ) A ij = x i y j Determinant (square matrices only): N det(a) = X (-1) i+j A ij M ij (A) j=1 M ij (A) = 0 A 11 A A 1(j-1) A 1(j+1)... A 1N A 21 A A 2(j-1) A 2(j+1)... A 2N det A (i-1)1 A (j-1)2... A (i-1)(j-1) A (i-1)(j+1)... A (i-1)n A (i+1)1 A (j+1)2... A (i+1)(j-1) A (i+1)(j+1)... A (i+1)n A N1 A N2... A N(j-1) A N(j+1)... A NN det(c) = c This minor is then calculated using the same recursive formula (equa 1 C A 11

11 Determinant (square matrices only): N det(a) = X (-1) i+j A ij M ij (A) Properties: j=1 If any row or column is zeros, det(a) =0 If any row or column is multiplied by a det(a c 1 A c 2 aa c 3... A c N) = a det(a) Swapping any row or column changes the sign det(at )=det(a) det(ab) = det(a)det(b) 12

12 Example: A = Calculate: det(a) How many operations to compute det(a) in general? N det(a) = X (-1) i+j A ij M ij (A) j=1 13

13 A = det(a) recursively takes O(N!) but MATLAB does it in O(N 3 ) 14

14 What are matrices? They represent transformations! Examples: y = Ax y = x1 /2 x 2 /

15 What are matrices? They represent transformations! Examples: y = Ax y = x 2 x

16 What are matrices? They represent transformations! Examples: y = Ax y = cos x1 - sin x 2 sin x 1 + cos x 2 cos - sin sin cos 17

17 What are matrices? They represent transformations! Examples: y = Ax

18 What are matrices? They represent transformations! If a transformation is unique, then it can be undone. The matrix is invertible: det(a) =0 6 A unique solution to the system of equations exists: x = A 1 y What happens if a transformation is just barely unique? 1 1+ A =

19 Matrices are maps between vector spaces! y = Ax x 2 R M A 2 R N M y 2 R N R M R N 21

20 Matrices are maps between vector spaces! y = Ax A 2 R N N x 2 R N y 2 R N A 1 2 R N N R N R N When a square matrix is invertible, there is a unique map back the other direction 22

21 Matrices are maps between vector spaces! y = Ax A 2 R N N x 2 R N y 2 R N R N R N When a square matrix is not invertible, the map is not unique or does not cover the entire vector space. 23

22 Matrices are maps between vector spaces! y = Ax A 2 R N N x 2 R N y 2 R N R N R N When a square matrix is not invertible, the map is not unique or does not cover the entire vector space. 24

23 Matrix norms: A 2 RN M x 2 R M Induced norms: kaxk kak p = max x p kxk p Among all vectors in RM, what is the maximum stretch caused by the matrix A? kyk 2 Example: let y = Ax then kak 2 = max x kxk 2 What is kak 1? kak 1 = max X A ij i j=1 N What is kak 1? kak 1 = max X A ij j i=1 25 M

24 Matrix norms: A 2 R N M x r 2 R M B 2 R M O What is kak 2? kak 2 = max A j (A T A) j A j (A T A) is an eigenvalue of A T A Properties: kak p > 0, kak p =0 only if A = 0 kcak p = c kak p kaxk p applekak p kxk p kabk p applekak p kbk p ka + Bk p applekak p + kbk p 26

25 Using matrix norms to estimate numerical error in solution of linear equations: Suppose: Ax = b, has exact solution: x = A 1 b If there is a small error in b, denoted 6b, how much of an error is produced in x? x + 6x = A -1 (b + 6b) 6x = A -1 6b Absolute error in x : k5xk p = ka 1 5bk p appleka 1 k p k5bk p Relative error in x : kbk p = kaxk p applekak p kxk p )kxk p > kbk p kak p k5xk p k5bk p applekak p ka -1 k p kxk p kbk p 27

26 Condition number: apple(a) =kak p ka 1 k p Measures how numerical error is magnified in solution of linear equations. Assume a unique solution exists, can we find it? (R.E. in answer) is bounded by (condition number) x (R.E. in data) log 10 apple(a) gives the number of lost digits Ill-conditioned means a large condition number Examples: apple(i) = apple

27 Condition number: apple(a) =kak p ka 1 k p Examples: Polynomial interpolation: X N y y i = j=1 a j x i j 1 y = Va x Vandermonde matrix: x 2 N 1 x 1... x 1 B 2 N B 1 x 2 x 2... x 2 V = B A 1 x N x 2 N... x N N apple(v) > N2 N, N 1 29

28 Condition number: Ax = b is ill-conditioned. What now? Rescale the equations: (D 1 A)x = D 1 b Rescale the unknowns: (AD 2 )(D 1 x)= b 2 Rescale both: (D 1 AD 2 )(D 1 x)= D 1 b 2 D 1 and D 2 are diagonal matrices An optimal rescaling exists: Braatz and Morari, SIAM J. Control and Optimization 32,

29 Condition number: Rescaling example: A = apple(a) D =, apple(da) The simplest solution is to rescale rows or columns by their maximum element 31

30 Preconditioning: Change the problem so it is easier to solve! Instead of solving: Ax = b Solve: (P 1AP 2 )(P 1 x)= P 1 b 2 P 1 left, P 2 right, preconditioner Perhaps the matrix P 1AP 2 has better properties: condition number structure sparsity pattern 33

31 MIT OpenCourseWare http ://ocw.mit.edu Numerical Methods Applied to Chemical Engineering Fall 2015 For information about citing these materials or our Terms of Use, visit: http ://ocw.mit.edu/terms.