Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems Tom Lyche University of Oslo Norway Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.1/33
Plan for the day 1. Notation 2. Splines Cubic spline interpolation Strictly diagonally dominant matrices LU-factorization of tridiagonal matrices 3. A two point boundary value problem The finite difference scheme Formulation as a matrix equation Weakly diagonally dominant tridiagonal matrices 4. An eigenvalue problem Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.2/33
Notation The set of natural numbers, integers, rational numbers, real numbers, and complex numbers are denoted by N, Z, Q, R, C, respectively. R n (C n ) is the set of n-tuples of real(complex) numbers which we will represent as column vectors. Thus x R n means x = x 1 x 2. x n, where x i R for i = 1,..., n. Row vectors are normally identified using the transpose operation. Thus if x R n then x is a column vector and x T is a row vector. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.3/33
Notation2 R m,n (C m,n ) is the set of m n matrices with real(complex) entries represented as a 11 a 12 a 1n a A = 21 a 22 a 2n.... a m1 a m2 a mn The entry in the ith row and jth column of a matrix A will be denoted by a i,j, a ij, A(i, j) or (A) i,j. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.4/33
Vector norms 1. x 1 = n j=1 x j, (the one-norm or l 1 -norm) 2. x 2 = ( n j=1 x j 2) 1/2, the two-norm, l2 -norm, or Euclidian norm) 3. x = max 1 j n x j, (the infinity-norm, l -norm, or max norm.) 4. x p := ( n j=1 x j p) 1/p, 1 p, (p-norm) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.5/33
Nonsingular matrix Definition 1. A square matrix A is said to be nonsingular if the only solution of the homogenous system Ax = 0 is x = 0. The matrix is singular if it is not nonsingular. Theorem 2. Suppose A R n,n (C n,n ). The linear system Ax = b has a unique solution x R n (C n ) for any b R n (C n ) if and only if the matrix A is nonsingular. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.6/33
The inverse matrix A square matrix is invertible if BA = AB = I for some square matrix B. B is unique and is called the inverse of A, denoted A 1 := B. Theorem 3. A square matrix is invertible if and only if it is nonsingular. Lemma 4. If A, B, C R n,n (C n,n ) with AB = C then C is nonsingular if and only if both A and B are nonsingular. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.7/33
Spline Definition 5. Given a positive integer K, a partition := [t 1,..., t k ] of R, with t 1 < t 2 < < t K, and integers r, d with r 1 and d 0. We say that a function s : R R is a C r -spline of degree d with knots if s(x) = s 0 (x), if x < t 1, s i (x), if t i x < t i+1 for i = 1,..., K 1, s K (x), if x t K, where each function s i is a polynomial of degree d and s (j) i 1 (t i) = s (j) i (t i ) for j = 0, 1,..., r and i = 1, 2,..., K. (2) We often call s a linear, quadratic, cubic, quartic, quintic C r -spline if d = 1, 2, 3, 4, 5, respectively. (1) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.8/33
A cubic C 2 -spline B(x) := s 0 (x) = 0, if x < 2, s 1 (x) = 1 6 (x + 2)3, if 2 x < 1, s 2 (x) = 1 2 x3 x 2 + 2 3, if 1 x < 0, s 3 (x) = 1 2 x3 x 2 + 2 3, if 0 x < 1, s 4 (x) = 1 6 (2 x)3, if 1 x < 2, s 5 (x) = 0, if x 2, (3) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.9/33
Spline s has r continuous derivatives on R, in other words s C r (R). The value at a knot t i is defined using the polynomial s i defining s to the right of t i. This means that the one sided(right) derivatives D j + s(x) := lim h 0 h>0 s(x + h) s(x) h exist for all integers j 0 and all x R. The one sided derivative of s equals the usual derivative of s for j r and all x R, and for j r for x /. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.10/33
Derivatives of a spline We have D j + s(x) = s (j) 0 (x), if x < t 1, s (j) i (x), if t i x < t i+1 for i = 1,..., K 1, K (x), if x t K, s (j) (4) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.11/33
Derivative 1 example 1 0.5 0 3 2 1 0 1 2 3 0.5 0 0.5 1 3 2 1 0 1 2 3 2 1 0 1 2 3 3 2 1 0 1 2 3 Figure 1: A C 2 -cubic spline(left); the first derivative (center); the second derivative (right). The first derivative of B is a C 1 -quadratic spline with knots [ 2, 1, 0, 1, 2], Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.12/33
Cubic spline interpolation Given data (x i, y i ) n i=1 in R2 with x 1 < x 2 < < x n. We shall construct a cubic C 2 -spline s : R R such that s(x i ) = y i for i = 1,..., n. The knots of the spline will be chosen at the interior data abscissas = {x 2,..., x n 1 }. The spline has n 1 unknown cubic polynomial pieces s 1 (x), if x < x 2, s(x) = s i (x), if x i x < x i+1 for i = 2,..., n 2, s n 1 (x), if x x n 1. (5) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.13/33
The interpolation problem We write each s i in the local power form s i (x) = s i3 (x x i ) 3 + s i2 (x x i ) 2 + s i1 (x x i ) + s i0, (6) For such a spline we have n + 2 degrees of freedom. Since we only have n interpolation points we need two extra conditons to specify the spline uniquely. We will only consider the clamped (first derivative) boundary conditions s (x 1 ) = y 1 and s (x n ) = y n, where y 1 and y n are given numbers. Thus we seek a cubic C 2 -spline s with knots x 2,..., x n 1 such that s(x i ) = y i, for i = 1,..., n and s (x i ) = y i for i = 1, n. (7) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.14/33
You just need a moment We can express the coefficients s ij in terms of the (at this point unknown) 2. derivatives (moments) as follows: s i3 = m i+1 m i 6h i where m i := s (x i ), for i = 1,..., n (8), s i2 = m i 2, s i1 = δ i h i ( ) 2mi +m i+1, si0 = y i 6 (9) h i := x i+1 x i, and δ i = y i+1 y i h i. (10) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.15/33
A linear system It remains to determine the second derivatives m i. For this we use the continuity of the first derivative of s. After some calculation d 1 h 1 m 1 b 1 h 1 d 2 h 2 m 2 b 2.......... =. h n 2 d n 1 h n 1 m n 1 b n 1 h n 1 d n m n b n, (11) where d i = 2 ( h i 1 + h i ), bi = 6 ( δ i δ i 1 ), i = 1,..., n, h 0 = h n = 0, and δ 0 = y 1, δ n = y n. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.16/33
Strict diagonal dominance Note that in the spline coefficient matrix the diagonal entry is strictly larger than the sum of the off-diagonal entries in that row. A matrix A C n,n is said to be strictly diagonally dominant if a ii > j i a ij for i = 1,..., n. Lemma 6. Suppose A C n,n is strictly diagonally dominant. Then the linear system Ax = b has a unique solution for any b C n. If b C n, then the solution x of Ax = b is bounded as follows: x := max x ( b j ) i max, (12) 1 i n 1 j n σ j where σ i := a ii j i a ij for i = 1,..., n. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.17/33
The spline linear system d 1 h 1 h 1 d 2 h 2......... h n 2 d n 1 h n 1 h n 1 d n m 1 m 2. m n 1 m n = b 1 b 2. b n 1 b n, where d i = 2 ( ) h i 1 + h i, i = 1,..., n. Strictly diagonally dominant and nonsingular Tridiagonal Solve linear system by LU-factorization Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.18/33
LU-factorization of a tridiagonal matri d 1 c 1 1 u a 2 d 2 c 2 1 c 1....... l 2 1........ =...... a n 1 d n 1 c n 1 l n 1 a n d n u n 1 Note that L has ones on the diagonal, and that A and U have the same super-diagonal. By equating entries we find d 1 = u 1, a k = l k u k 1, d k = l k c k 1 + u k, k = 2, 3,..., n. c n 1 u n (13). Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.19/33
The Algorithm A = LU (LU-factorization) Ly = b (forward substitution) Ux = y (backward substituion) u 1 = d 1, l k = a k u k 1, u k = d k l k c k 1, k = 2, 3,..., n. y 1 = b 1, y k = b k l k y k 1, k = 2, 3,..., n, x n = y n /d n, x k = (y k c k x k+1 )/u k, k = n 1,..., 2, 1. This process is well defined if A is strictly diagonally dominant. The number of arithmetic operations (flops) is O(n). Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.20/33
s (1976) = y 1981 y 1976 1981 1976 and s (2006) = y 2006 y 2005 2006 2005 2.4 x 105 2.2 2 1.8 1.6 1.4 1.2 1 0.8 0.6 1975 1980 1985 1990 1995 2000 2005 2010 Figure 2: Foreign citizens in Norway 1976-2006. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.21/33
A boundary value problem u (x) = f(x), x [0, 1], (14) u(0) = 0, u(1) = 0. f is a given continuous function on [0, 1]. Solve using a finite difference method Choose positive integer m Define discretization parameter h := 1/(m + 1) Replace the interval [0, 1] by grid points x j := jh for j = 0, 1,..., m + 1. Replace derivative with finite difference approximation u (x) u(x h) 2u(x)+u(x+h) h 2. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.22/33
Tridiagonalt linear system u (x) = f(x), u(0) = u(1) = 0 v j u(x j ) for all j v j 1 +2v j v j+1 h 2 = f(jh), j = 1,..., m, v 0 = v m+1 = 0 This leads to a linear system T [v 1,..., v m ] T = h 2 [f(h),..., f(mh)] T T := tridiag m ( 1, 2, 1) = 2 1 0 1 2 1 0......... 0 1 2 1 0 1 2 R m,m (15) Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.23/33
Weak diagonal dominance Definition 7. The tridiagonal matrix A = A := tridiag(a i, d i, c i ) = d 1 c 1 a 2 d 2 c 2......... a n 1 d n 1 c n 1 a n d n is weakly diagonally dominant if d 1 > c 1, d n > a n, d k a k + c k, k = 2, 3,..., n 1. T is weakly diagonally dominant. The matrix A 1 = also weakly diagonally dominant, but it is singular. [ 2 1 0 0 0 0 0 1 2 (16) ] is Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.24/33
Irreducibility Definition 8. A tridiagonal matrix tridiag(a i, d i, c i ) is irreducible if and only if all the a i, c i are nonzero. A matrix which is not irreducible is called reducible. Clearly the matrix T is irreducible, while the matrix A 1 = [ 2 1 0 0 0 0 0 1 2 ] is reducible and singular. Theorem 9. Suppose A is tridiagonal, weakly diagonally dominant, and irreducible. Then A is nonsingular and has a unique LU-factorization. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.25/33
LU of T T = LU, where 1 0 0 1 2 1.... L = 0 2 3 1.............. 0 0 0 m 1 m 1, U = 2 1 0 0 3 0 2 1.............. 0... m m 1. 1 0 0 m+1 m (17) We can now solve T v = h 2 f by a forward and backward substitution. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.26/33
An eigenvalue problem Consider a horizontal beam of length L located between 0 and L on the x-axis of the plane. We assume that the beam is fixed at x = 0 and x = L A force F is applied at (L, 0) in the direction towards the origin. This situation can be modeled by the boundary value problem Ry (x) = F y(x), y(0) = y(l) = 0 y(x) is the vertical displacement of the beam at x R is a constant defined by the rigidity of the beam. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.27/33
The transformed system We can transform the problem to the unit interval [0, 1] by considering the function u : [0, 1] R given by u(t) := y(tl). Since u (t) = L 2 y (tl), the problem Ry (x) = F y(x), y(0) = y(l) = 0 becomes u (t) = Ku(t), u(0) = u(1) = 0, K := F L2 R Eigenvalue problem When F is increased it will reach a critical value where the beam will buckle and maybe break. This corresponds to the smallest eigenvalue of u = Ku. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.28/33
The smallest eigenvalue With u(t) = sin πt we find u = π 2 u K = π 2 gives a nonzero solution. It can be shown that this is the smallest eigenvalue. Since π 2 = K = F L2 R we find F = π2 R/L 2. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.29/33
Eigenvalue approximation A discrete version of u = Ku is v j 1 + 2v j v j+1 h 2 = Kv j, j = 1,..., m, h = 1 m + 1 v 0 = v m+1 = 0, v j u(jh) for j = 0,..., m + 1. If we define λ := h 2 K then we obtain the matrix eigenvalue problem T v = λv v = [v 1,..., v m ] T, and T := tridiag m ( 1, 2, 1) The problem is to determine the smallest eigenvalue λ min of T. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.30/33
The eigenvalues of tridiag(a, b, a) for a, b R C := tridiag(a, b, a) = b a 0... 0 a b a... 0......... 0... a b a 0... a b R m,m. Cs j = λ j s j for j = 1,..., m, λ j = b + 2a cos(jπh), with h = 1/(m + 1) s j = [sin (jπh), sin (2jπh),..., sin (mjπh)] T The eigenvalues are distinct The eigenvectors are orthogonal s T j s k = 1 2h δ j,k Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.31/33
The smallest eigenvalue of T Normally we need a numerical method to find λ min. λ min = 2 2 cos (πh) = 4 sin 2 (πh/2) Since λ = h 2 K = h2 F L 2 R we can solve for F to obtain F = 4 sin2 (πh/2)r h 2 L 2. For small h this is a good approximation to the value we computed above. π 2 R L 2 F = 4 sin2 (πh/2)r h 2 L 2 = π2 R L 2 + O(h 2 ). Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.32/33
Summary Studied 3 problems Each leads to a tridiagonal matrix Introduced the concepts; diagonal dominance (strict, weak) Irreducibility for a tridiagonal matrix Introduced the 2. derivative matrix T Found the eigenvalues and eigenvector of T Eigenvectors of T are orthogonal. Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.33/33