Pseudospectra and Nonnormal Dynamical Systems

Transcription

1 Pseudospectra and Nonnormal Dynamical Systems Mark Embree and Russell Carden Computational and Applied Mathematics Rice University Houston, Texas ELGERSBURG MARCH 1

2 Overview of the Course These lectures describe modern tools for the spectral analysis of dynamical systems We shall cover a mix of theory, computation, and applications Lecture 1: Introduction to Nonnormality and Pseudospectra Lecture : Functions of Matrices Lecture : Toeplitz Matrices and Model Reduction Lecture : Model Reduction, Numerical Algorithms, Differential Operators Lecture : Discretization, Extensions, Applications

3 Outline for Today Lecture : Functions of Matrices and Model Reduction Toeplitz matrices Model Reduction: Balanced Truncation Nonnormality and Lyapunov Equations Model Reduction: Moment Matching

4 (a) Toeplitz Matrices

5 Pseudospectra Recall the example that began our investigation of pseudospectra yesterday Example Compute eigenvalues of three similar 1 1 matrices using MATLAB s eig / 1/ 1/ 1/

6 Toeplitz Matrices Consider the pseudospectra of the 1 1 matrix in the middle of the last slide, A = tridiag(,, 1/) A is diagonalizable (it has distinct eigenvalues), but Bauer Fike is useless here: κ(v) =

7 Jordan Blocks We ve already analyzed pseudospectra of Jordan blocks near λ for small ε > Here we want to investigate the entire pseudospectrum for larger ε S n = 1 1 Near the eigenvalue, the resolvent norm grows with dimension n; outside the unit disk, the resolvent norm does not seem to get big We would like to prove this

8 Jordan Blocks We ve already analyzed pseudospectra of Jordan blocks near λ for small ε > Here we want to investigate the entire pseudospectrum for larger ε S n = 1 1 Near the eigenvalue, the resolvent norm grows with dimension n; outside the unit disk, the resolvent norm does not seem to get big We would like to prove this

9 Jordan Blocks We ve already analyzed pseudospectra of Jordan blocks near λ for small ε > Here we want to investigate the entire pseudospectrum for larger ε S n = 1 1 Near the eigenvalue, the resolvent norm grows with dimension n; outside the unit disk, the resolvent norm does not seem to get big We would like to prove this

10 Jordan Blocks/Shift Operator Consider the generalization of the Jordan block to the domain l (N) = {(x 1, x, ) : X x j < } j=1 The shift operator S on l (N) is defined as S(x 1, x, ) = (x, x, )

11 Jordan Blocks/Shift Operator Consider the generalization of the Jordan block to the domain l (N) = {(x 1, x, ) : X x j < } j=1 The shift operator S on l (N) is defined as S(x 1, x, ) = (x, x, ) In particular, S(1, z, z, ) = (z, z, z, ) = z(1, z, z, ) So if (1, z, z, ) l (N), then z σ(s)

12 Jordan Blocks/Shift Operator Consider the generalization of the Jordan block to the domain l (N) = {(x 1, x, ) : X x j < } j=1 The shift operator S on l (N) is defined as S(x 1, x, ) = (x, x, ) In particular, S(1, z, z, ) = (z, z, z, ) = z(1, z, z, ) So if (1, z, z, ) l (N), then z σ(s) If z < 1, then So, X z j 1 1 = 1 z < j=1 {z C : z < 1} σ(s)

13 Jordan Blocks/Shift Operator S(x 1, x, ) = (x, x, ) We have seen that {z C : z < 1} σ(s) Observe that and so The spectrum is closed, so S = sup Sx = 1, x =1 σ(s) {z C : z 1} σ(a) = {z C : z 1} For any finite dimensional n n Jordan block S n, σ(s n) = {}

14 Jordan Blocks/Shift Operator S(x 1, x, ) = (x, x, ) We have seen that {z C : z < 1} σ(s) Observe that and so The spectrum is closed, so S = sup Sx = 1, x =1 σ(s) {z C : z 1} σ(a) = {z C : z 1} For any finite dimensional n n Jordan block S n, σ(s n) = {} So the S n S strongly, but there is a discontinuity in the spectrum: σ(s n) σ(s)

15 Pseudospectra of Jordan Blocks Pseudospectra resolve this unpleasant discontinuity Recall the eigenvectors (1, z, z, ) for S Truncate this vector to length n, and apply it to S n: z z n z n 1 = z z z n 1

16 Pseudospectra of Jordan Blocks Pseudospectra resolve this unpleasant discontinuity Recall the eigenvectors (1, z, z, ) for S Truncate this vector to length n, and apply it to S n: z z n z n 1 = z z z n 1 = z 1 z z n z n 1 z n

17 Pseudospectra of Jordan Blocks Pseudospectra resolve this unpleasant discontinuity Recall the eigenvectors (1, z, z, ) for S Truncate this vector to length n, and apply it to S n: z z n z n 1 = z z z n 1 = z 1 z z n z n 1 z n Hence, S nx z x = z n, so for all ε > z n x = z n p 1 z n p 1 z, z σ ε(s n) We conclude that for fixed z < 1, the resolvent norm (z S n) 1 grows exponentially with n

18 Upper Triangular Toeplitz Matrices Consider an upper triangular Toeplitz matrix giving the matrix with constant diagonals containing the Laurent coefficients: a a 1 a a A n = a1 a C n n a a 1 a Definition (Symbol, Symbol Curve) Toeplitz matrices are described by their symbol a with Taylor expansion X a(z) = a k z k Call the image of the unit circle T under a the symbol curve, a(t) k=

19 Pseudospectra of Upper Triangular Toeplitz Matrices Apply the same approximate eigenvector we used for the Jordan block: a a 1 a a n 1 a a1 a a a 1 a 1 z z n z n 1 = P n 1 k= a k z k P n 1 k= a kz k+1 a z n + a 1z n 1 a z n 1 If the matrix has fixed bandwidth b n, (ie, a k = for k > b), then a a b a ab a a 1 z z n z n 1 = P b k= a k z k P b k= a kz k+1 P b 1 k= a kz k+n b a z n 1

20 Pseudospectra of Upper Triangular Toeplitz Matrices a a b a ab a a 1 z z n z n 1 = P b k= a k z k P b k= a kz k+1 P b 1 k= a kz k+n b a z n 1 = bx k= a k z k 1 z z n z n 1 a b z n P b k=1 a kz n 1+k

21 Pseudospectra of Upper Triangular Toeplitz Matrices a a b a ab a a 1 z z n z n 1 = P b k= a k z k P b k= a kz k+1 P b 1 k= a kz k+n b a z n 1 = bx k= a k z k 1 z z n z n 1 a b z n P b k=1 a kz n 1+k Hence A nx a(z)x gets very small in size as n In fact, this reveals the spectrum of the infinite Toeplitz operator on l (N)

22 Spectrum of Toeplitz Operators on l (Z) For the Toeplitz operator (semi-infinite matrix) A with the same symbol: a a b a ab a 1 z z = P b k= a k z k P b k= a kz k+1 P b k= a kz k+ = a(z) 1 z z

23 Spectrum of Toeplitz Operators on l (Z) For the Toeplitz operator (semi-infinite matrix) A with the same symbol: a a b a ab a 1 z z = P b k= a k z k P b k= a kz k+1 P b k= a kz k+ = a(z) 1 z z Thus a(z) σ(a) for all z < 1 In fact, one can show that for this banded upper triangular symbol, σ(a ) = {a(z) : z 1}

24 Spectrum of Toeplitz Operators on l (Z) For the Toeplitz operator (semi-infinite matrix) A with the same symbol: a a b a ab a 1 z z = P b k= a k z k P b k= a kz k+1 P b k= a kz k+ = a(z) 1 z z Thus a(z) σ(a) for all z < 1 In fact, one can show that for this banded upper triangular symbol, σ(a ) = {a(z) : z 1} The calculation on the last slide guarantees that for any ε >, there exists N > such that if n > N, σ(a ) σ ε(a n)

25 Symbols of Upper Triangular Toeplitz Matrices Symbol curves for banded upper triangular Toeplitz matrices a(z) = z + z a(z) = z + z + z + z a(z) = z + z z a(z) = z + z a(z) = z + z

26 General Toeplitz Matrices More generally, the dense Toeplitz matrix a a 1 a a 1 a A n = a a1 a a 1 a a 1 a a 1 a C n n follows the same terminology Definition (Symbol, Symbol Curve) Toeplitz matrices are described by their symbol a with Laurent expansion X a(z) = a k z k k= Call the image of the unit circle T under a the symbol curve, a(t)

27 Spectrum of a General Toeplitz Operator Theorem (Spectrum of a Toeplitz Operator) Suppose the Toeplitz operator A : l (N) l (N) has a symbol that is continuous Then σ(a ) = a(t ) {all points a(t ) encloses with nonzero winding number} Due variously to: Wintner; Gohberg; Krein; Calderón, Spitzer, & Widom See Albrecht Böttcher and colleagues for many more details

28 Spectrum of a General Toeplitz Operator Theorem (Spectrum of a Toeplitz Operator) Suppose the Toeplitz operator A : l (N) l (N) has a symbol that is continuous Then σ(a ) = a(t ) {all points a(t ) encloses with nonzero winding number} Due variously to: Wintner; Gohberg; Krein; Calderón, Spitzer, & Widom See Albrecht Böttcher and colleagues for many more details a(t)

29 Spectrum of a General Toeplitz Operator Theorem (Spectrum of a Toeplitz Operator) Suppose the Toeplitz operator A : l (N) l (N) has a symbol that is continuous Then σ(a ) = a(t ) {all points a(t ) encloses with nonzero winding number} Due variously to: Wintner; Gohberg; Krein; Calderón, Spitzer, & Widom See Albrecht Böttcher and colleagues for many more details a(t) σ(a )

30 Spectrum of a General Toeplitz Matrix What can be said of the eigenvalues of a finite-dimensional Toeplitz matrix? Theorem (Limiting Spectrum of Finite Toeplitz Matrices) Consider the family of banded Toeplitz matrices {A n} n N with upper bandwidth b and lower bandwidth d For any fixed λ C, label the roots ζ 1,, ζ b+d of the polynomial z d (a(z) λ) by increasing modulus If ζ d = ζ d+1, then λ lim n σ(an) This result, proved by [Schmidt & Spitzer, 19], shows that in general: lim σ(an) σ(a ) n

31 Spectrum of a General Toeplitz Matrix What can be said of the eigenvalues of a finite-dimensional Toeplitz matrix? Theorem (Limiting Spectrum of Finite Toeplitz Matrices) Consider the family of banded Toeplitz matrices {A n} n N with upper bandwidth b and lower bandwidth d For any fixed λ C, label the roots ζ 1,, ζ b+d of the polynomial z d (a(z) λ) by increasing modulus If ζ d = ζ d+1, then λ lim n σ(an) This result, proved by [Schmidt & Spitzer, 19], shows that in general: lim σ(an) σ(a ) n a(t) lim σ(an) n

32 Spectrum of a General Toeplitz Matrix: Example a(t) lim σ(an) n σ(a )

33 Spectrum of a General Toeplitz Matrix: Example a(t) lim σ(an) n σ(a 1)

34 Spectrum of a General Toeplitz Matrix: Example a(t) lim σ(an) n σ(a )

35 Spectrum of a General Toeplitz Matrix: Example a(t) lim σ(an) n σ(a )

36 Spectrum of a General Toeplitz Matrix: Example a(t) lim σ(an) n σ(a 8)

37 Pseudospectra of Toeplitz Matrices Theorem (Landau; Reichel and Trefethen; Böttcher) Let a(z) = P M k= M a kz k be the symbol of a banded Toeplitz operator The pseudospectra of A n converge to the pseudospectra of the Toeplitz operator on l (N) as n Let z C have nonzero winding number wrt the symbol curve a(t ) (z A n) 1 grows exponentially in n For all ε >, z σ ε(a n) for all n sufficiently large n = 1 n = n = n = 18

38 Pseudospectra of Toeplitz Matrices symbol curve σ ε(a 1)

39 Pseudospectra of Toeplitz Matrices symbol curve σ ε(a )

40 Hermitian Toeplitz Matrices We have seen large pseudospectra arise for generic Toeplitz matrices But what about Hermitian Toeplitz matrices? For example, the matrix has symbol a(z) = z 1 + z, so a(e i θ ) = e i θ + e i θ = cos(θ) [, ] R

41 Hermitian Toeplitz Matrices We have seen large pseudospectra arise for generic Toeplitz matrices But what about Hermitian Toeplitz matrices? For example, the matrix has symbol a(z) = z 1 + z, so a(e i θ ) = e i θ + e i θ = cos(θ) [, ] R In general, Hermitian symbols have a k = a k, so a(e i θ ) = a + X 1 k= k=1 a k e i k θ + X a k e i k θ k=1 X X X = a + a k e i k θ + a k e i k θ = a + Re(a k e i k θ ) R As n, eigenvalues of A n distribute according to Szegő s theorem k=1 k=1

42 Tridiagonal Toeplitz Matrices Consider the case of a tridiagonal Toeplitz matrix, β γ α β A n = γ Cn n α β with symbol a(z) = α z + β + γz The symbol curve is the ellipse a(t) = {z T : α z + β + γz}

43 Eigenvectors of Tridiagonal Toeplitz Matrices A = tridiag(1,, 1), n = Normal matrix: orthogonal eigenvectors

44 Eigenvectors of Tridiagonal Toeplitz Matrices A = tridiag(,, 1/), n = Nonnormal matrix: non-orthogonal eigenvectors

45 Circulant Matrices and Laurent Operators In contrast, consider the circulant matrix a a 1 a a 1 a 1 a a C n = a1 a a 1 a a 1 a 1 a a 1 a C n n

46 Circulant Matrices and Laurent Operators In contrast, consider the circulant matrix a a 1 a a 1 a 1 a a C n = a1 a a 1 a a 1 a 1 a a 1 a C n n C n is normal for all symbols C n is diagonalized by the Discrete Fourier Transform matrix σ(c n) = {a(z) : z T n}, where T n := {e kπi/n, k =,, n 1}: ie, σ(c n) comprises the image of the nth roots of unity under the symbol Infinite dimensional generalization: Laurent operators C on l (Z) (doubly-infinite matrices) with spectrum σ(c ) = a(t)

47 Circulant Matrix and Laurent Operators σ(c ) = a(t) σ(c 1) σ(c ) σ(c )

48 Piecewise Continuous Symbols It is possible for the symbol a(z) = X k= a k z k to, eg, have a jump discontinuity This compromises the exponential growth of the norm of the resolvent [Böttcher, E, Trefethen, ] For example, take a(e i θ ) = θe i θ, a symbol studied by [Basor & Morrison, 199] a(t) σ(a ) σ ε(a ) σ ε(a 1)

49 Twisted Toeplitz Matrices A twisted Toeplitz matrix is a Toeplitz-like matrix with varying coefficients [Trefethen & Chapman, ] For example, for x j = πj/n, set A n = x 1 1 x1 x 1 xn 1 x n The symbol now depends on two variables: a(x, z) = x + 1 xz The pseudospectra resemble those of standard Toeplitz matrices, but the (pseudo)-eigenvectors have an entirely different character

50 Twisted Toeplitz Matrices For x j = πj/n, set A n = x 1 1 x1 x 1 xn 1 x n Pseudospectra of σ ε(a 1)

51 Eigenvectors of a Twisted Toeplitz Matrix Eigenvectors form wave packets for twisted Toeplitz matrices Pseudoeigenvectors for z σ ε(a) have a similar form Eigenvectors of A for a(x, z) = x + 1 xz

52 Looking Forward to Differential Operators Where do Toeplitz (and twisted Toeplitz) matrices come from? Numerous applications be we will highlight just one of them: discretization of differential operators Consider the steady state convection diffusion equation Lu = f, where Lu = u + c u posed over for x [, 1] with u() = u(1) = Fix a discretization parameter n Approximate the problem on a simple grid {x j } n+1 j= h = 1/(n + 1): x j = j h with spacing Replace the first and second derivatives with second-order accurate formulas on the grid: u (x j ) = u(x j+1) u(x j+1 ) h + O(h ) u (x j ) = u(x j 1) u(x j ) + u(x j+1 ) h + O(h )

53 Finite Difference Discretizations Now let u j u(x j ) with u = u n+1 =, so that Approximate Lu = f, with u (x j ) u j+1 u j+1 h u (x j ) u j 1 u j + u j+1 h Lu = u + c u, u() = u(1) =, via 1 h 1 + ch/ 1 ch/ 1 + ch/ 1 ch/ u 1 u u n = f (x 1) f (x ) f (x n) Label these components: A nu = f A n is Toeplitz for this constant-coefficient differential operator

54 Finite Difference Discretizations A n := 1 h 1 + ch/ 1 ch/ 1 + ch/ 1 ch/ Notice that the symbol of A n depends on n (recall h = 1/(n + 1)): 1 a n(z) = h c ««z h h h + c «z h

55 Finite Difference Discretizations A n := 1 h 1 + ch/ 1 ch/ 1 + ch/ 1 ch/ Notice that the symbol of A n depends on n (recall h = 1/(n + 1)): 1 a n(z) = h c ««z h h h + c «z h For all n, the symbol curve a n(t) is an ellipse

56 Finite Difference Discretizations A n := 1 h 1 + ch/ 1 ch/ 1 + ch/ 1 ch/ Notice that the symbol of A n depends on n (recall h = 1/(n + 1)): 1 a n(z) = h c ««z h h h + c «z h For all n, the symbol curve a n(t) is an ellipse If c = (no convection), then a n(t) is a real line segment: A n and L are both self-adjoint, hence normal If c, the eigenvalues of A n will be real, but the pseudospectra can be far from these eigenvalues

57 Finite Difference Discretizations A n := 1 h 1 + ch/ 1 ch/ 1 + ch/ 1 ch/ Notice that the symbol of A n depends on n (recall h = 1/(n + 1)): 1 a n(z) = h c ««z h h h + c «z h For all n, the symbol curve a n(t) is an ellipse If c = (no convection), then a n(t) is a real line segment: A n and L are both self-adjoint, hence normal If c, the eigenvalues of A n will be real, but the pseudospectra can be far from these eigenvalues Rightmost part of σ ε(a n) approximates the corresponding part of σ ε(l)

58 Finite Differences: Symbol Curves Symbol curves a n(t) for n = 1,,

59 Finite Differences: Symbol Curves Symbol curves a n(t) for n = 1,,

60 Finite Differences: Approximate Pseudospectra Rightmost part of σ ε(a n) for n =

61 Finite Differences: Approximate Pseudospectra Rightmost part of σ ε(a n) for n = 18

62 Finite Differences: Approximate Pseudospectra Rightmost part of σ ε(a n) for n =

63 Finite Differences: Approximate Pseudospectra Rightmost part of σ ε(a n) for n = 1

64 (b) Balanced Truncation Model Reduction

65 Balanced Truncation Model Reduction Consider the single-input, single-output (SISO) linear dynamical system: ẋ(t) = Ax(t) + bu(t) y(t) = cx(t), A C n n, b, c T C n We assume that A is stable: α(a) < We wish to reduce the dimension of the dynamical system by projecting onto well-chosen subspaces Balanced truncation: Change basis to match states that are easy to reach and easy to observe, then project onto that prominent subspace

66 Controllability and Observability Gramians To guage the observability of an initial state x = bx, measure the energy in its output (when there is no input, u = ): Then Z y(t) dt = y(t) = ce ta bx Z = bx Z bx e ta c ce ta bx dt e ta c ce ta dt bx =: bx Qbx

67 Controllability and Observability Gramians To guage the observability of an initial state x = bx, measure the energy in its output (when there is no input, u = ): Then Z y(t) dt = y(t) = ce ta bx Z = bx Z bx e ta c ce ta bx dt e ta c ce ta dt bx =: bx Qbx Similarly, we measure the controllability by the total input energy required to steer x = to a target state bx as t The special form of u that drives the system to bx with minimal energy satisfies Z u(t) dt = bx Z 1bx e ta bb e dt ta =: bx P 1 bx

68 Controllability and Observability Gramians To guage the observability of an initial state x = bx, measure the energy in its output (when there is no input, u = ): Then Z y(t) dt = y(t) = ce ta bx Z = bx Z bx e ta c ce ta bx dt e ta c ce ta dt bx =: bx Qbx Similarly, we measure the controllability by the total input energy required to steer x = to a target state bx as t The special form of u that drives the system to bx with minimal energy satisfies Z u(t) dt = bx Z 1bx e ta bb e dt ta =: bx P 1 bx Thus we have the infinite controllability and observability gramians P and Q: P := Z See, eg, [Antoulas, ] e ta bb e ta dt, Q := Z e ta c ce ta dt

69 Balanced Truncation Model Reduction The gramians P := Z e ta bb e ta dt, Q := Z e ta c ce ta dt (Hermitian positive definite, for a controllable and observable stable system) can be determined by solving the Lyapunov equations see the next lecture If x =, the minimum energy of u required to drive x to state bx is bx P 1 bx Starting from x = bx with u(t), the energy of output y is bx Qbx bx P 1 bx: bx is hard to reach if it is rich in the lowest modes of P bx Q bx: bx is hard to observe if it is rich in the lowest modes of Q Balanced truncation transforms the state space coordinate system to make these two gramians the same, then it truncates the lowest modes

70 Balanced Truncation Model Reduction Consider a generic coordinate transformation, for S invertible: (Sx) (t) = (SAS 1 )(Sx(t)) + (Sb)u(t) y(t) = (cs 1 )(Sx(t)) + du(t), (Sx)() = Sx With this transformation, the controllability and observability gramians are bp = SPS, b Q = S QS 1 For balancing, we seek S so that b P = b Q are diagonal

71 Balanced Truncation Model Reduction Consider a generic coordinate transformation, for S invertible: (Sx) (t) = (SAS 1 )(Sx(t)) + (Sb)u(t) y(t) = (cs 1 )(Sx(t)) + du(t), (Sx)() = Sx With this transformation, the controllability and observability gramians are bp = SPS, b Q = S QS 1 For balancing, we seek S so that b P = b Q are diagonal Observation (How does nonnormality affect balancing?) σ ε/κ(s) (SAS 1 ) σ ε(a) σ εκ(s) (SAS 1 ) The choice of internal coordinates will affect P, Q,

72 Balanced Truncation Model Reduction Consider a generic coordinate transformation, for S invertible: (Sx) (t) = (SAS 1 )(Sx(t)) + (Sb)u(t) y(t) = (cs 1 )(Sx(t)) + du(t), (Sx)() = Sx With this transformation, the controllability and observability gramians are bp = SPS, b Q = S QS 1 For balancing, we seek S so that b P = b Q are diagonal Observation (How does nonnormality affect balancing?) σ ε/κ(s) (SAS 1 ) σ ε(a) σ εκ(s) (SAS 1 ) The choice of internal coordinates will affect P, Q, but not the Hankel singular values: b P b Q = SPQS 1, and not the transfer function: (cs 1 )(z SAS 1 ) 1 (Sb) = d + c(z A) 1 b, and not the system moments: (cs 1 )(Sb) = cb, (cs 1 )(SAS 1 )(Sb) = cab,