Toeplitz matrices. Niranjan U N. May 12, NITK, Surathkal. Definition Toeplitz theory Computational aspects References

Size: px

Start display at page:

Download "Toeplitz matrices. Niranjan U N. May 12, NITK, Surathkal. Definition Toeplitz theory Computational aspects References"

Shanna Barrett
5 years ago
Views:

1 Toeplitz matrices Niranjan U N NITK, Surathkal May 12, 2010 Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

2 1 Definition Toeplitz matrix Circulant matrix 2 Toeplitz theory Boundedness Compactness and self-adjointness Szegö s theorem 3 Computational aspects Product of Toeplitz matrix and a vector Operations Simulation 4 References Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

3 Toeplitz matrix Structure c d e b c d a b c So n n Toeplitz matrix T n is described by the vector t = [t 0, t 1,..., t 2n 2 ], i.e., a Toeplitz matrix is sparse in information but dense in structure. Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

4 Toeplitz matrix Structure c d e b c d a b c So n n Toeplitz matrix T n is described by the vector t = [t 0, t 1,..., t 2n 2 ], i.e., a Toeplitz matrix is sparse in information but dense in structure. Identity matrix is also Toeplitz Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

5 Circulant matrix Structure a b c Right circulant matrix c a b b c a Hankel and left circulant matrices Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

6 Circulant matrix Structure a b c Right circulant matrix c a b b c a Hankel and left circulant matrices Vandermonde matrix: evaluating polynomial Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

7 Circulant matrix Structure a b c Right circulant matrix c a b b c a Hankel and left circulant matrices Vandermonde matrix: evaluating polynomial Sylvester matrix square [T 1 T 2 ] of 2 polynomials: Determinant is aka resultant; if res=0 then common root. Some observations Every square Hankel matrix is symmetric Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

8 Circulant matrix Structure a b c Right circulant matrix c a b b c a Hankel and left circulant matrices Vandermonde matrix: evaluating polynomial Sylvester matrix square [T 1 T 2 ] of 2 polynomials: Determinant is aka resultant; if res=0 then common root. Some observations Every square Hankel matrix is symmetric All the row and column sums of a left as well as right circulant matrices are constant Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

9 Boundedness Theorem (Toeplitz) a 0 a 1 a 2... A = a 1 a 0 a 1... a 2 a 1 a A defines a bounded operator on l 2 iff {a n } are the Fourier coefficients of a L (T), Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

10 Boundedness Theorem (Toeplitz) a 0 a 1 a 2... A = a 1 a 0 a 1... a 2 a 1 a A defines a bounded operator on l 2 iff {a n } are the Fourier coefficients of a L (T), a n = 1 2π 2π 0 a(e iθ )e inθ dθ The norm of the operator is given by Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

11 Boundedness Theorem (Toeplitz) a 0 a 1 a 2... A = a 1 a 0 a 1... a 2 a 1 a A defines a bounded operator on l 2 iff {a n } are the Fourier coefficients of a L (T), a n = 1 2π 2π 0 a(e iθ )e inθ dθ The norm of the operator is given by a := ess sup t T a(t) Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

12 Boundedness Proof. Lebesgue spaces on the complex unit circle Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

13 Boundedness Proof. Lebesgue spaces on the complex unit circle Multiplication operator M(a) : L 2 L 2, f af is bounded iff a L and norm is given above Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

14 Boundedness Proof. Lebesgue spaces on the complex unit circle Multiplication operator M(a) : L 2 L 2, f af is bounded iff a L and norm is given above Orthonormal basis of L 2 e n (t) = 1 2π t n, t T Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

15 Boundedness Proof. Lebesgue spaces on the complex unit circle Multiplication operator M(a) : L 2 L 2, f af is bounded iff a L and norm is given above Orthonormal basis of L 2 e n (t) = 1 2π t n, t T a 0 a 1 a 2... L(a) =... a 1 a 0 a a 2 a 1 a Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

16 Boundedness Proof. Lebesgue spaces on the complex unit circle Multiplication operator M(a) : L 2 L 2, f af is bounded iff a L and norm is given above Orthonormal basis of L 2 e n (t) = 1 2π t n, t T a 0 a 1 a 2... L(a) =... a 1 a 0 a a 2 a 1 a A L(a) = a Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

17 Boundedness Let S n : (x k ) (..., 0, x n,..., x 0, x 1,...) Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

18 Boundedness Let S n : (x k ) (..., 0, x n,..., x 0, x 1,...) A = S n L(a)S n = A = S n L(a)S n Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

19 Boundedness Let S n : (x k ) (..., 0, x n,..., x 0, x 1,...) A = S n L(a)S n = A = S n L(a)S n Strong convergence: not only in inner product but also norm. Here to S n identity operator. Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

20 Boundedness Let S n : (x k ) (..., 0, x n,..., x 0, x 1,...) A = S n L(a)S n = A = S n L(a)S n Strong convergence: not only in inner product but also norm. Here to S n identity operator. L(a) lim inf n S n L(a)S n Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

21 Boundedness Let S n : (x k ) (..., 0, x n,..., x 0, x 1,...) A = S n L(a)S n = A = S n L(a)S n Strong convergence: not only in inner product but also norm. Here to S n identity operator. L(a) lim inf n S n L(a)S n Compare previous relations. So, L(a) A Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

22 Boundedness Let S n : (x k ) (..., 0, x n,..., x 0, x 1,...) A = S n L(a)S n = A = S n L(a)S n Strong convergence: not only in inner product but also norm. Here to S n identity operator. L(a) lim inf n S n L(a)S n Compare previous relations. So, L(a) A A is bounded iff a L = A = a Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

23 Boundedness Let S n : (x k ) (..., 0, x n,..., x 0, x 1,...) A = S n L(a)S n = A = S n L(a)S n Strong convergence: not only in inner product but also norm. Here to S n identity operator. L(a) lim inf n S n L(a)S n Compare previous relations. So, L(a) A A is bounded iff a L = A = a SYMBOL is a: Since the Fourier coefficients are unique, the equivalent class of L containing a is unique. Operator induced by A is T (a) Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

24 Compactness and self-adjointness Theorem The only compact Toeplitz operator is the zero operator Proof. Let a L. Suppose T(a) is compact. Let Q n : (x 0, x 1,...) (0,..., 0, x n, x n+1,...). Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

25 Compactness and self-adjointness Theorem The only compact Toeplitz operator is the zero operator Proof. Let a L. Suppose T(a) is compact. Let Q n : (x 0, x 1,...) (0,..., 0, x n, x n+1,...). This converges strongly to 0. So, like the argument in boundedness proof, compression Q n T (a)q n has same matrix as T(a). So norms are equal. T (a) = 0 Theorem Toeplitz operator is self-adjoint iff symbol is real valued. C* algebra (Banach algebra with involution a a = a 2 ) Fredholm operators Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

26 Szegö s theorem Szegö s asymtotic eigenvalue distribution theorem lim n [ 1 n Σn 1 k=0 F (τ n,k)] = 1 2π 2π 0 F (f (λ))dλ Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

27 Szegö s theorem Szegö s asymtotic eigenvalue distribution theorem lim n [ 1 n Σn 1 k=0 F (τ n,k)] = 1 2π 2π 0 F (f (λ))dλ Approximation Approximate asymptotic properties of sequences of Toeplitz matrices by sequences asymptotically equivalent of circulant matrices. Also, perturbed Toeplitz matrices and Toeplitz-like. Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

28 Szegö s theorem Szegö s asymtotic eigenvalue distribution theorem lim n [ 1 n Σn 1 k=0 F (τ n,k)] = 1 2π 2π 0 F (f (λ))dλ Approximation Approximate asymptotic properties of sequences of Toeplitz matrices by sequences asymptotically equivalent of circulant matrices. Also, perturbed Toeplitz matrices and Toeplitz-like. Some applications Differential entropy rate of a Gaussian process Shannon rate-distortion function of a Gaussian process CFD - 1 D diffusion discretization; Symmetric tridiagonal matrix - recursion for characteristic polynomial(sturm sequence). Householder s method for 3rd order real symmetric matrix. Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

29 Szegö s theorem Recursion formula is a 11 a 12 0 A = a 12 a 22 a 23 0 a 23 a 33 φ k (λ) = (a kk λ)φ k 1 (λ) a 2 k 1,k φ k 2(λ) φ 0 (λ) = 1 Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

30 Product of Toeplitz matrix and a vector Preliminaries Evaluate / add polynomial: Θ(n), Horner s rule Convolution: c k = Σ k j=0 a jb j k Multiplication using FFT algorithm: Θ(nlogn) Interpolation (pt to coeff) is reverse of evaluation (coeff to pt) Lagrange for pt to coeff:a(x) = Σ n 1 k=0 y k Π j k(x x j ) Π j k (x k x j ) Toeplitz vector = finding coeff in O(log n) time using O(n log n) Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

31 Operations Inverse Use FFT (odd and even points) Inverting triangular Toeplitz matrices is equivalent to polynomial division Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

32 Operations Inverse Use FFT (odd and even points) Inverting triangular Toeplitz matrices is equivalent to polynomial division [ ] T1 0 T =. By induction, it can be proved that T 2 T [ 1 T 1 T 1 ] = 1 0 T1 1 T 2 T1 1 T1 1. Recursively compute first column of T 1. Then T1 1 T 2 T1 1 by Toeplitz vector product. T (n) = T (n/2) + O(logn) and W (n) = W (n/2) + O(nlogn) O(log 2 n) time using O(n log n) operations on EREW PRAM For circulant O(logn) time using O(n log n) operations Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

33 Operations Newton s iteration (I λa)(i + λa λ k A k ) = I λ k+1 A k+1 (I λa)(i + λa λ k A k ) = Imodλ k+1 Algorithm Set X 0 = I, B = I λa till log(n + 1) do X i = (2I X i 1 B)X i 1 Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

34 Operations Newton s iteration (I λa)(i + λa λ k A k ) = I λ k+1 A k+1 (I λa)(i + λa λ k A k ) = Imodλ k+1 Algorithm Set X 0 = I, B = I λa till log(n + 1) do X i = (2I X i 1 B)X i 1 Other algorithms Trench s algorithm for inverse Levinson s algorithm for Toeplitz linear equations Durbin s algorithm Yule-Walker problem Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

35 Simulation Software LinBox is a high-performance C++ library for exact computational linear algebra over the integers and finite fields. Matlab Maple Mathematica Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

36 Toepltiz and circulant matrices: A review. Robert M Gray Matrix computations. Golub and Van Loan Introduction to parallel algorithms. Joseph JaJa Toeplitz matrices and asymtotic LA. Bottcher and Grudsky CFD. Anderson Numerical analysis. Sastry Niranjan U N (NITK, Surathkal) Linear Algebra May 12, / 15

Numerical Methods in Matrix Computations

Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices