Lecture 1 INF-MAT : Chapter 2. Examples of Linear Systems

Transcription

1 Lecture 1 INF-MAT : Chapter 2. Examples of Linear Systems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo August 26, 2010

2 Notation The set of natural numbers, integers, rational numbers, real numbers, and complex numbers are denoted by N, Z, Q, R, C, respectively. R n (C n ) is the set of n-tuples of real(complex) numbers which we will represent as column vectors. Thus x R n means x 1 x 2 x =., x n where x i R(C) for i = 1,..., n. Row vectors are normally identified using the transpose operation. Thus if x R n then x is a column vector and x T is a row vector.

3 Notation2 R m,n (C m,n ) is the set of m n matrices with real(complex) elements represented as a 11 a 12 a 1n a 21 a 22 a 2n A =.... a m1 a m2 a mn The element in the ith row and jth column of a matrix A will be denoted by a i,j, a ij, A(i, j) or (A) i,j.

4 Column Partition of a Matrix Product Matrix vector product. a1j x j Ax =. = a 1j. x j = amj x j a mj In particular Ae j = a :j. n x j a :j Product (AB)e j = A(Be j ) = Ab :j so AB = [Ab :1,..., Ab :n ] AB is partitioned by columns Partition by rows a T 1: B a T 2: AB = B. a T m:b j=1

5 Partitioned Matrices A rectangular matrix A can be partitioned into submatrices by drawing horizontal lines between selected rows and[ vertical ] lines between selected columns. For example, A = can be partitioned as [ ] A11 A (i) 12 = 4 5 6, (ii) [ ] a A 21 A :1, a :2, a :3 = a T 1: (iii) a T 2: = a T 3: , (iv) [ ] A 11, A 12 = The submatrices in a partition is often referred to as blocks and a partitioned matrix is sometimes called a block matrix..

6 Block Multiplication 1 B 1 B 2 p m A AB 1 AB 2 m AB = [Ab :1,..., Ab :r, Ab :r+1..., Ab :n ] = [AB 1, AB 2 ].

7 Block Multiplication 2 m A 1 A 2 B 1 s m A 1 B 1 A 2 B 2 B 2 p s p s p (AB) ij = a ik b kj = a ik b kj + a ik b kj j=1 j=1 j=s+1 = (A 1 B 1 ) ij + (A 2 B 2 ) ij = (A 1 B 1 + A 2 B 2 ) ij.

8 The General Case If A 11 A 1s B 11 B 1q A =.., B =.., A p1 A ps B s1 B sq and if all the matrix products in s C ij = A ik B kj, k=1 i = 1,..., p, j = 1,..., q are well defined then C 11 C 1q AB =... C p1 C pq

9 Products of Triangular Matrices a 11 a 12 a 1n b 11 b 12 b 1n c 11 c 12 c 1n 0 a 22 a 2n 0 b 22 b 2n.... = 0 c 22 c 2n a nn 0 0 b nn 0 0 c nn Lemma The product C = AB = (c ij ) of two upper(lower) triangular matrices A = (a ij ) and B = (b ij ) is upper(lower) triangular with diagonal elements c ii = a ii b ii for all i. Proof. Exercise.

10 Block Triangular Matrices Lemma Suppose [ ] A11 A A = 12 0 A 22 where A, A 11 and A 22 are square matrices. Then A is nonsingular if and only if both A 11 and A 22 are nonsingular. In that case [ A 1 A 1 = 11 A 1 11 A 12A 1 ] 22 0 A 1 (1) 22

11 Proof If A 11 and A 12 are nonsingular then [ A 1 11 A 1 11 A 12A 1 ] [ ] 22 A11 A 12 0 A 1 = 22 0 A 22 and A is nonsingular with the indicated inverse. [ ] I 0 = I 0 I

12 Proof Conversely, let B be the inverse of the nonsingular matrix A. We partition B conformally with A and have [ ] [ ] [ ] B11 B BA = 12 A11 A 12 I 0 = = I B 21 B 22 0 A 22 0 I Using block multiplication we find B 11 A 11 = I, B 21 A 11 = 0, B 21 A 12 + B 22 A 22 = I. The first equation implies that A 11 is invertible, this in turn implies that B 21 = 0 in the second equation, and then the third equation simplifies to B 22 A 22 = I. We conclude that also A 22 is invertible.

13 The Inverse Consider now a triangular matrix. Lemma An upper (lower) triangular matrix A = [a ij ] R n,n is nonsingular if and only if the diagonal elements a ii, i = 1,..., n are nonzero. In that case the inverse is upper (lower) triangular with diagonal elements a 1 ii, i = 1,..., n. Proof: We use induction on n. The result holds for n = 1: The 1-by-1 matrix A = (a 11 ) is invertible if and only if a 11 0 and in that case A 1 = (a 1 11 ). Suppose the result holds for n = k and let A = R k+1,k+1 be upper triangular.

14 Proof We partition A in the form [ ] Ak a A = k 0 a k+1,k+1 and note that A k R k,k is upper triangular. By Lemma 1.1 A is nonsingular if and only if A k and (a k+1,k+1 ) are nonsingular and in that case [ A A 1 1 k A 1 k = a ] ka 1 k+1,k+1 0 a 1. k+1,k+1 By the induction hypothesis A k is nonsingular if and only if the diagonal elements a 11,..., a kk of A k are nonzero and in that case A 1 k is upper triangular with diagonal elements a 1 ii, i = 1,..., k. The result for A follows.

15 Unit Triangular Matrices A matrix is unit triangular if it is triangular with 1 s on the diagonal. Lemma For a unit upper(lower) triangular matrix A R n,n : 1. A is invertible and the inverse is unit upper(lower) triangular. 2. The product of two unit upper(lower) triangular matrices is unit upper(lower) triangular. Proof. 1. follows from the inverse Lemma while the above Lemma implies 2.

16 Two Point Boundary Value Problem u (x) = f (x), x [0, 1] u(0) = 0, u(1) = 0, where f is a given continuous function on [0, 1]. Unique solution Numerical solution: finite difference method m N, h := 1/(m + 1), v j u(jh), j = 0,..., m + 1 Discrete system: Tv = v j 1 +2v j v j+1 = h 2 f (jh), j = 1,..., m, v 0 = v m+1 = v 1 v 2. v m 1 v m = h 2 f (h) f (2h). f ((m 1)h) f (mh) =: b.

17 LU Factorization d 1 c 1 a 2 d 2 c 2 1 r 1 c l = a n 1 d n 1 c n 1 l a n d n 1 n r n 1 c n 1 r n. d 1 = r 1, a k = l k r k 1, d k = l k c k 1 + r k, k = 2, 3,..., n. Algorithm (trifactor) 1. function [l,r]=trifactor(a,d,c) 2. r=d; l=d; 3. for k=2:length(d) 4. l(k)=a(k)/r(k-1); 5. r(k)=d(k)-l(k)*c(k-1); 6. end

18 Solving Tridiagonal Ax = b by LU Factorization Ax = L(Rx) = b y 1 = b 1, y k = b k l k y k 1, k = 2, 3,..., n, x n = y n /r n, x k = (y k c k x k+1 )/r k, k = n 1,..., 2, 1 Algorithm (trisolve) 1. function x=trisolve(l,r,c,b) 2. x=b; n=length(b); 3. for k=2:n 4. x(k)=b(k)-l(k)*x(k-1); 5. end 6. x(n)=x(n)/r(n); 7. for k=n-1:-1:1 8. x(k)=(x(k)-c(k)*x(k+1))/r(k); 9. end

19 O(n) Complexity l k = a k /r k 1, r k = d k l k c k 1, k = 2,..., n. n 1 divisions, n 1 multiplications, and n 1 subtractions. A total of 3n 3 flops (floating point arithmetic operations). y 1 = b 1, y k = b k l k y k 1, k = 2, 3,..., n, 2n 2 flops x n = y n /r n, x k = (y k c k x k+1 )/r k, k = n 1,..., 2, 1. 3n 2 flops A total of 3n 3 + 2n 2 + 3n 2 = 8n 7 flops. # flops is O(n). Gaussian elimination on a full matrix requires O(n 3 ) flops.

20 Diagonal Dominance The matrix A = [a ij ] C n,n is diagonally dominant if a ii j i a ij, i = 1,..., n. (2) It is strictly diagonally dominant if strict inequality holds for i = 1,..., n. T is diagonally dominant. Is it nonsingular? A square matrix A is singular if Ax = 0 for a nonzero vector x. A diagonally dominant matrix can be singular. [ ] [ ] A 1 = and A 2 = are both diagonally dominant and singular.

21 Diagonal Dominance and Nonsingularity d 1 c 1 a 2 d 2 c 2 A := a n 1 d n 1 c n 1 a n d n Theorem Suppose A = tridiag(a i, d i, c i ) C n,n is tridiagonal and diagonally dominant. If in addition d 1 > c 1 and a i 0 for i = 2,..., n 1, then A has a unique LU factorization. If in addition d n 0, then A is nonsingular. T is nonsingular. We can solve the system Tv = f in O(n) arithmetic operations using trifactor and trisolve.

22 Proof l k = a k /r k 1, r k = d k l k c k 1, k = 2,..., n. The matrix A has an LU factorization if the r k s are nonzero for k = 1,..., n 1. We show by induction on k that r k > c k for k = 1,..., n 1. Clearly r 1 = d 1 > c 1 Suppose c k 1 / r k 1 < 1 for some 2 k n 1 r k = d k l k c k 1 = d k a kc k 1 r k 1 d k a k c k 1 r k 1. a k 0 implies r k > d k a k c k. Thus r k > c k 0, for k = 1,..., n 1 and an LU factorization exists. It is unique since any LU factorization must satisfy the deriving equations. If d n 0 then r n > 0 regarless of wether a n is zero or nonzero. L and R have nonzero diagonal elements so the product A = LR is nonsingular.

23 The inverse of T We could solve the system Tv = b by using the inverse T 1 of T and simply compute the matrix vector product v = T 1 b. n=6: T 1 = All elements in T 1 are nonzero and the calculation of T 1 b requires O(n 2 ) flops. Using trifactor and trisolve only requires O(n) flops. So using the inverse is not a good idea.

24 An Interpolation Problem Consider a different problem. Given an integer n 2, x values x = [x 1,..., x n ] T with a := x 1 < < x n =: b. y values y = [y 1,..., y n ] T, derivative values σ a, σ b. Find a function g : [a, b] R such that g(x i ) = y i, for i = 1,..., n, g (a) = σ a, g (b) = σ b. The conditions g (a) = σ a, g (b) = σ b are called 1. derivative (clamped, Hermite) boundary conditions.

25 Special case n = 2: Cubic Hermite Interpolation f (x) = x 4, g(x) = 4x 3 4x 2, g is a cubic polynomial, g(0) = f (0), g(2) = f (2), g (0) = f (0), g (2) = f (2).

26 Special case: polynomial interpolation to Arctan f (x) = Arctan(10x) + π/2, 1 x 1. n + 2 interpolation conditions. g is a polynomial interpolant to f of degree n + 1 = 15 x i = 1 + 2(i 1)/(n 1), i = 1,..., n = 14. Large oscillations near the ends of the range.

27 Cubic Spline Definition Let n 3, and a = x 1 < x 2 <... < x n = b. A function g : [a, b] R of the form p 1 (x), if a x < x 2, p 2 (x), if x 2 x < x 3, g(x) :=. p n 2 (x), if x n 2 x < x n 1, p n 1 (x), if x n 1 x b, (3) is called a cubic spline with knots x = [x 1,..., x n ] T provided (i) Each p i is a polynomial of degree 3. (ii) p i 1 (x i ) = p i (x i ), p i 1 (x i) = p i (x i), p i 1 (x i) = p i (x i), i = 2,..., n 1.

28 Cubic Spline Interpolant Given x values x = [x 1,..., x n ] T with a := x 1 < < x n =: b. y values y = [y 1,..., y n ] T, Derivative values σ a, σ b. A cubic spline with knots x that satisfies g(x i ) = y i, for i = 1,..., n, g (a) = σ a, g (b) = σ b is called a cubic spline interpolant. Each of the n 1 cubic polynomials p i has 4 coefficients and there are 3 continuity conditions at each of the n 2 interior knots. Thus, a cubic spline appears to have 4(n 1) 3(n 2) = n + 2 degrees of freedom. This is the same as the number of interpolation conditions

29 Special case: cubic spline interpolation to Arctan f (x) = Arctan(10x) + π/2, 1 x 1. Compute cubic spline interpolant g for n = 14, x i = 1 + 2(i 1)/(n 1), i = 1,..., n. g is composed of 13 polynomial pieces.

30 PP Representation We will represent the polynomial pieces in shifted power form: p i (x) = c 1i + c 2i (x x i ) + c 3i (x x i ) 2 + c 4i (x x i ) 3. In general the pp form of a cubic spline g with knots x 1,..., x n is c 11 c 12 c 1,k x s := [x 1,..., x k ], C := c 21 c 22 c 2,k c 31 c 32 c 3,k R4,k, c 41 c 42 c 4,k shifts x s, coefficients: C. k = n 1.

31 Determining the interpolant Given a = x 1 <... < x n = b, y i R, i = 1,..., n, and any numbers s 1,..., s n. Suppose the pp form x s, C = [c ji ] of g is given by x T s = [x 1,..., x n 1 ] and c 1i := y i, c 2i := s i, c 3i := (3δ i 2s i s i+1 )/h i, c 4i := ( 2δ i + s i + s i+1 )/h 2 i, (4) where Then h i = x i+1 x i, g(x i ) = y i, g (x i ) = s i for i = 1,..., n. δ i = y i+1 y i h i. (5) p i 1 (x i ) = p i (x i ), p i 1 (x i) = p i (x i), i = 2,..., n 1.

32 The C 2 Conditions For a cubic spline we need in addition p i 1 (x i) = p i (x i), i = 2,..., n 1. p i (x) = c 1i + c 2i (x x i ) + c 3i (x x i ) 2 + c 4i (x x i ) 3. p i (x) = 2c 3,i + 6c 4,i (x x i ) p i (x i) = 2c 3,i, p i 1 (x i) = 2c 3,i 1 + 6c 4,i 1 h i 1. (1 0 = h i 1 h i 2 p i 1(x i ) 1 2 p i (x i ) ) ( ) = h i 1 h i c3,i 1 + 3c 4,i 1 h i 1 c i,3 = h i (3δ i 1 2s i 1 s i ) + 3h i ( 2δ i 1 + s i 1 + s i ) h i 1 (3δ i 2s i s i+1 ) = h i s i 1 + 2(h i 1 + h i )s i + h i 1 s i+1 3(h i δ i 1 + h i 1 δ i ). Rearranging we obtain h i s i 1 +2(h i 1 +h i )s i +h i 1 s i+1 = 3(h i δ i 1 +h i 1 δ i ), i = 2,..., n 1.

33 A Linear System h i s i 1 +2(h i 1 +h i )s i +h i 1 s i+1 = 3(h i δ i 1 +h i 1 δ i ), Multiplying the last expression by 2/(h i 1 + h i ) leads to the equations λ i = i = 2,..., n 1 λ i s i 1 + 4s i + µ i s i+1 = β i, i = 2,..., n 1, (6) 2h i h i 1 + h i, µ i = 2h i 1 h i 1 + h i, h i = x i+1 x i, β i = 3(λ i δ i 1 + µ i δ i ), δ i = y i+1 y i. h i Obtain the linear system 1 0 s 1 λ 2 4 µ 2 s 2 Ns = = λ n 1 4 µ n 1 s n s n σ a β 2. β n 1 σ b (7) =: b. (8)

34 The matrix N 1 0 λ 2 4 µ 2 N = λ n 1 4 µ n N is tridiagonal and strictly diagonally dominant. The algorithms trifactor and trisolve can be used to find s. N is nonsingular. The cubic spline interpolant exists and is unique.

35 Finding g Solve the linear systen Ns = b, Compute the pp representation, O(n) flops.

36 Uniform knots Assume h i = h > 0, all i. λ i = 2h i h i 1 +h i = 1, µ i = 2h i 1 h i 1 +h i = 1. β i = 3(λ i δ i 1 + µ i δ i ) = 3((y i y i 1 )/h + (y i+1 y i )/h = 3(y i+1 y i 1 )/h. 1 0 s 1 σ a s 2 3(y 3 y 1 )/h Ns = = s n 1 3(y n y n 2 )/h 0 1 s n σ b

37 Example, f (x) = x 4 x = [0, 1, 2] T, y = [0, 1, 16] T, σ a = 0, σ b = s s 2 = s 3 32 s 1 = 0, s 3 = 32, s 2 = 4. c 1,1 = y 1 = 0, c 2,1 = s 1 = 0, c 3,1 = (3δ 1 2s 1 s 2 )/h = = 1, c 4,1 = ( 2δ 1 + s 1 + s 2 )/h 2 = = 2, c 1,2 = y 2 = 1, c 2,2 = s 2 = 4, c 3,2 = (3δ 2 2s 2 s 3 )/h = = 5, c 4,2 = ( 2δ 2 + s 2 + s 3 )/h 2 = = 6, p 1 (x) = x 2 + 2x 3. p 2 (x) = 1 + 4(x 1) + 5(x 1) 2 + 6(x 1) 3 = x 13x 2 + 6x 3.

38 Smooth Piecewise Polynomial, f (x) = x 4 16 g(x) := { p 1 (x) = x 2 + 2x 3, if 0 x < 1, p 2 (x) = x 13x 2 + 6x 3, if 1 x 2,

39 Compute g(x), but where is x? Algorithm (findsubintervals) Given shifts x s = [x 1,..., x k ] and a real number r, an integer i is computed so that i = 1 if r < x(2) i = k if r x k and x i r < x i+1 otherwise. If r is a vector then a vector i is computed, such that the jth component of i gives the location of the jth component of r. 1. function i=findsubintervals(xs,r) 2. k=length(xs); m=length(r); 3. xs(1)=min(r)-1; 4. [sorted,j] = sort([xs(:) r(:) ]); 5. i = find(j>k)-(1:m);

40 Spline Evaluation Algorithm (cubppeval) Given a pp-representation (x s, C) of a cubic spline g together with x-values r R m. The vector q = g(r) is computed. 1. function q=cubppeval(xs,c,r) 2. i=findsubintervals(xs,r); q=r; 3. for j=1:length(r) 4. k=i(j); t=r(j)-xs(k); 5. q(j)=[1 t t 2 t 3 ]*C(:,k); 6. end

41 Plotting a Spline in MATLAB x=[0 1 2]; n=length(x); h=(x(n)-x(1))/199; r=x(1):h:x(n)+h/2; xs=[0 1]; C=[0 1; 0 4; -1 5; 2 6]; q=cubppeval(xs,c,r) plot(r,q);

42 Plot 16

43 The Physical Spline A cubic spline interpolant minimizes g (x) 2 dx over all C 2 interpolants. It is a mathematical model of a physical spline.