Banded symmetric Toeplitz matrices: where linear algebra borrows from difference equations. William F. Trench Professor Emeritus Trinity University

Size: px

Start display at page:

Download "Banded symmetric Toeplitz matrices: where linear algebra borrows from difference equations. William F. Trench Professor Emeritus Trinity University"

Gary Beverly Ward
5 years ago
Views:

1 Banded symmetric Toeplitz matrices: where linear algebra borrows from difference equations William F. Trench Professor Emeritus Trinity University This is a lecture presented the the Trinity University Matheematics Seminar during the 9 Fall semester.

2 A Toeplitz matrix, named after the German mathematician Otto Toeplitz (88-9), is of the form T D Œt r s n r;sd. (It s ok, and convenient for Toeplitz matrices, to number rows and columns from to n.) A symmetric Toeplitz matrix is of the form T n D Œt jr sj n. For example, T D r;sd t t t t t t t t t t t t t t t t t t t t t t t t t is a symmetric Toeplitz matrix. We will assume that t, t,..., t k are all real numbers. From your linear algebra course you know that a symmetric matrix with real entries has real eigenvalues and is always diagonalizable; that is, T n has real eigenvalues and n linearly independent eigenvectors.

3 A Toeplitz matrix is said to be banded if there is an integer d < n such that t` D if ` > d. In this case, we say that T has bandwidth d. For example, T D t t t t t t t t t t t t t t t t t t t is a banded symmetric Toeplitz matrix with bandwidth.

4 The eigenvalue problem for very large (n can be in the thousands!) symmetric banded Toeplitz matrices pops up in many statistical problems. In your linear algebra course you learned to solve the eigenvalue problem for a matrix A by factoring its characteristic polynomial p./ D det.a I/: Sorry, that s impossible for big matrices. In general there is no computationally useful way to obtain the characteristic polynomial of a large symmetric matrix (or any other large matrix). All methods for finding a single eigenvalue of an arbitrary n n symmetric matrix carry a computational cost (it s called complexity) proportional to n. So, if you double the size of the matrix you make the problem of obtaining a single eigenvalue eight times more difficult. However, the situation is different for banded symmetric Toeplitz matrices.

5 Let s start with the simplest case: d D. t t t t t T n D t t t : : : : ::: : : : t t t t t nn with t. This is a symmetric tridiagonal Toeplitz matrix. A vector x D x x : x n is a -eigenvector of T n if and only if t x C t x D x t x j C t x C t x j C D x j ; j n ; t x n C t x n D x n which we can rewrite as t x j C.t /x j C t x j C D ; j n; (a homogeneous difference equation) if we define x D x nc D (boundary conditions).

6 The characteristic polynomial of the difference equation is t x j C.t /x j C t x j C D (DE) p. I / D t C.t / Ct D t..//..//i thus, p..// D p..// D. (We don t know./ and./ yet; be patient.) If we let x j D c j./ C c j./ where c and c are arbitrary constants, then the left side of (DE) equals c j p..// C c j p..// D for any choice of c and c. Now let s work on the boundary conditions. Since x D if and only if c D c, x j D c. j./ j.//: Now x nc D if and only if../=.// nc D, which is true if and only if qi./ D q exp n C and./ D q exp qi ; n C where exp.i/ D e i D cos C i sin, q D ; : : : ; n and q is to be determined. (Letting q D does not produce an eigenvector because if./ D./) then x j D for all j ).

7 Taking note that are q possibilities, the eigenvectors have the form x q x q D x q : x n ;q where x jq D c. j. q/ j. q//; qi q./ D q exp ; and q./ D q exp n C so x jq D c exp jqi n C exp jqi n C D ci sin jq n C : Since c is arbitrary, it makes sense to let c D = q i. (Don t worry that maybe q D ; we ll see that it isn t.) Then x jq D sin jq n C ; j n : ALL SYMMETRIC TRIDIAGONAL TOEPLITZ MATRI- CES HAVE THE SAME EIGENVECTORS! qi ; n C

8 Now let s find q, the eigenvalue associated with q. t C.t q / C t D t..//..// which equals t../ C.// C././ Since./ D q exp t C.t qi n C q / Ct D t qi and./ D q exp ; n C q q cos C q : n C Equating coefficients on the two sides yields q D and q q D t C t cos ; q n: n C

9 Now suppose d >. To see where we re going, a nonzero vector x x x D x x x is a -eigenvector of if and only if T D t t t t t t t t t t t t t t t t t t t

10 t t t t t t t t t t t t t t t t t t t x x x x x D x x x x x ; or, equivalently, t x C t x C t x D x t x C t x C t x C t x D x t x C t x C t x C t x C t x D x t x C t x C t x C t x D x t x C t x C t x D x ;

11 (repeated for clarity) t x C t x C t x D x t x C t x C t x C t x D x t x C t x C t x C t x C t x D x or, equivalently, t x C t x C t x C t x D x t x C t x C t x D x ; t x C t x C t x C t x C t x D x t x C t x C t x C t x C t x D x t x C t x C t x C t x C t x D x t x C t x C t x C t x C t x D x t x C t x C t x C t x C t x D x if we impose the boundary conditions Better yet, x D x D x D x D : X `D t j`j x`cr D x r ; r :

12 (repeated for clarity) X `D with boundary conditions t j`j x`cr D x r ; r ; x D x D x D x D : For the general case where T n D Œt jr sj n r;sd with t` D if ` > d, the eigenvalue problem can be written as `D d subject to t j`j x`cr D x r ; r n ; (DE) x r D ; d r ; n r n C d : (BC) Eqn. (DE) is a difference equation and the conditions in (BC) are called boundary conditions. Obviously, (DE) and (BC) both hold for any if x r D for d r ncd. However, that s not interesting, since an eigenvector must be nonzero. Finding the values of for which (DE) has nonzero solutions that satisfy (BC) is a boundary value problem.

13 The characteristic polynomial of the difference equation is `D d t j`j x`cr D x r ; r n ; (DE) P. ; / D `D d t j`j ` The zeros of P. ; / are continuous functions of and, since P. ; / D P.= ; /, they occur in reciprocal pairs :../; =.//; : : : ;. d./; = d.//: It can be shown (don t you hate that?) that these zeros are distinct except for at most finitely many bad values of. We ll assume that none of these bad values are actually eigenvalues of T n. (This is a pretty safe bet.) Then (DE) holds if x r D sd a s rs./ C b s r s./ ; d r n C d ; where a,..., a d and b,..., b d are arbitrary constants.

14 PROOF. Recall that P. ; / D `D d t j`j `. If x r D sd a s rs./ C b s r s./ ; d r ncd ; then `D d t j`j x`cr x r D D C `D d sd sd t j`j d X sd sd a s `Cr s C b s s ` r a s rs./ C b s s r./ a s b s r s./ `D `D d t j`j `s./ A t j`j ` s./ A D sd a s rs./p. s./; / C b s r s./p.= s./; / D (look at the top of this page) for all r. Note that a,..., a d and b,..., b d are completely arbitrary up to this point.

15 Now we must choose them to satisfy the boundary conditions x r D ; d r ; n r n C d I (BC) that is, sd a s rs./ C b s r s./ D ; for d r and n r n C d. For example, if d D, we must have././././ a././././ n./ n./ n./ n./ a b nc./ nc./ n./ n./ b D :

16 For clarity,././././ a././././ n./ n./ n./ n./ a b D : nc./ nc./ n./ n./ b Let P./ D./././ ; Q./ D./ n R n./ D n./ nc nc ;./ n S n./ D n./ n n ;./ a b a D ; b D : a b Then the boundary conditions are satisfied if and only if P./ Q./ a D : R n./ S n./ b

17 In general, let P./ D Œ s r./ d r;sd ; Q./ D Œ rs./ ; R n./ D Œ ncr s./ ; S n./ D Œ s ncr./ a b a D a : and b D b : : a d b d Then the boundary conditions are satisfied if and only if P./ Q./ a D : (S) R n./ S n./ b Let D n./ D ˇ P./ Q./ R n./ S n./ ˇ (determinant). An eigenvector of T n must be a nonzero vector. Since (S) a has only the trivial solution D if D b n./, it follows that is an eigenvalue of T n if and only if D n./ D. For ways to find the zeros of D n./, see my papers RP-,,, and 8. Since D n./ doesn t become more complicated as n increases, the difficulty of finding individual eigenvalues of T n is independent of n.

18 We can take this a little further. The eigenvectors of a symmetric Toeplitz matrix have a special property that I haven t mentioned. To identify this property, let J n be the flip matrix, which has s on its secondary diagonal and s elsewhere. For example, Note that J D D J D D I : : In general, J n D I n; that is, J n is its own inverse.

19 Multiplying a vector by J n reverses ( flips ) the components of the vector. For example, if x x x D x x ; x then J x D In general, if x D x x : x n x n x x x x x D then J nx D x x x x x x n x n : x x :

20 We say that a vector x is symmetric if J n x D x, or skewsymmetric if J n x D x. Adding symmetric vectors produces a symmetric vector, and multiplying a symmetric vector by a real number produces a symmetric vector; hence, the symmetric vectors in R n form a subspace of R n. Similarly, the skew-symmetric vectors form a subspace of R n. If n D m then each of these subspaces has dimension m. If n D m C then the subspace of symmetric vectors has dimension m C and the subspace of skew symmetric vectors has dimension m. The zero vector is the only vector that is both symmetric and skew-symmetric. For example, if n D then and form a basis for the subspace of symmetric vectors, while and form a basis for the subspace of skew-symmetric vectors.

21 If n D then ; ; and form a basis for the subspace of symmetric vectors, while and form a basis for the subspace of skew-symmetric vectors. In general, if n D m then the subspace of symmetric vectors and the subspace of skew-symmetric vectors are both m-dimensional. If n D m C then the subspace of symmetric vectors is.m C /-dimensional and the subspace of skew-symmetric vectors is m-dimensional.

22 Multiplying a matrix on the left by J n reverses the rows of the matrix, so t t t t t t t t t t J T D t t t t t t t t t t t t t t t t t t t t t t t t t D t t t t t t t t t t t t t t t

23 Multiplying a matrix on the right by J n reverses the columns of the matrix, so J T J D.J T /J t t t t t t t t t t D t t t t t t t t t t t t t t t t t t t t t t t t t D t t t t t t t t t t t t t t t D T : In general, J n T n J n D T n. What does this tell us about the eigenvectors of T n?

24 First, suppose is an eigenvalue of T n with multiplicity one, so the associated eigenspace is one-dimensional, and suppose that T n x D x with x. Then J n T n x D J n x. Since J n J n D I n, it follows that.j n T n J n /.J n x/ D J n x. Therefore T n.j n x/ D.J n x/, because J n T n J n D T n. Since the -eigenspace of T n is one-dimensonal, it follows that J n x D cx for some constant c. Therefore, since kj n xk D kxk (that is, x and J n x have the same length), it follows that c D ; that is, x is either symmetric or skew-symmetric. The situation is more complicated if is a repeated eigenvalue of T n with multiplicity k. However, it can be shown that if k D ` then the -eigenspace of T n has a basis consisting of ` symmetric and ` skew-symmetric vectors, while if k D ` C then the -eigenspace has a basis consistng of either ` symmetric and ` C skewsymmetric eigenvectors or ` C skew-symmetric and `- symmetric eigenvectors. In any case, if n D k then T n has k symmetric linearly independent eigenvectors and k linearly independent skew-symmetric eigenvectors, while if n D k C then T n has k C linearly independent symmetric eigenvectors and k linearly independent skewsymmetric vectors.

25 Recall that the components of the eigenvectors of T n satisfy `D d subject to t j`j x`cr D x r ; r n ; (DE) x r D ; d r ; n r n C d ; (BC) and are therefore of the form x r D sd a s rs./ C b s r s./ ; d r n C d : (A) However, we now know that we can assume at the outset that the eigenvectors of T n are either symmetric, which means that x n rc D x r, or skew-symmetric, which means that x n rc D x r. So let s build this into (A) at the start!

26 For clarity x r D sd a s rs./ C b s r s./ ; d r n C d : To get a symmetric eigenvector, let b s D a s nc s./ so x r D sd a s rs./ C n rc s./ : Since n.n r C / D r, x n rc D x r. As for the boundary conditions x r D ; d r ; n r n C d ; (BC) it is enough to require that x r D for d r, since this and the equality x n rc D x r implies that x r D for n r n C d. Therefore, is an eigenvalue with an associated symmetric eigenvector if and only if i det h s r./ C ncrc d s./ D : r;sd

27 For clarity x r D sd a s rs./ C b s r s./ ; d r n C d : To get a skew-symmetric eigenvector, let b s D a s nc s./ so x r D sd a s rs./ n rc s./ : Therefore, is an eigenvalue with an associated skew-symmetric eigenvector if and only if i det h r d./ D : s./ ncrc s r;sd

Remark By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.

Remark By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero. Sec 6 Eigenvalues and Eigenvectors Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called an eigenvalue of A if there is a nontrivial