Nonlinear Eigenvalue Problems: Interpolatory Algorithms and Transient Dynamics

Transcription

1 Nonlinear Eigenvalue Problems: Interpolatory Algorithms and Transient Dynamics Mark Embree Virginia Tech with Serkan Gugercin Jonathan Baker, Michael Brennan, Alexander Grimm Virginia Tech SIAM Conference on Applied Linear Algebra Hong Kong May 218 Thanks to: International Linear Algebra Society (ILAS) US National Science Foundation DMS

2 a talk in two parts... rational interpolation for nlevps Rational / Loewner techniques for nonlinear eigenvalue problems, motivated by algorithms from model reduction. Structure Preserving Rational Interpolation Data-Driven Rational Interpolation Matrix Pencils Minimal Realization via Rational Contour Integrals transients for delay equations Scalar delay equations: a case-study for how one can apply pseudospectra techniques to analyze the transient behavior of a dynamical system. Finite dimensional nonlinear problem infinite dimensional linear problem Pseudospectral theory applies to the linear problem, but the choice of norm is important

3 nonlinear eigenvalue problems: the final frontier? problem typical # eigenvalues standard eigenvalue problem (A λi)v = n generalized eigenvalue problem (A λe)v = n quadratic eigenvalue problem (K + λd + λ 2 M)v = 2n polynomial eigenvalue problem nonlinear eigenvalue problem ( d k= λk A k )v = dn ( d k= f k(λ)a k )v =

4 nonlinear eigenvalue problems: the final frontier? problem typical # eigenvalues standard eigenvalue problem (A λi)v = n generalized eigenvalue problem (A λe)v = n quadratic eigenvalue problem (K + λd + λ 2 M)v = 2n polynomial eigenvalue problem nonlinear eigenvalue problem ( d k= λk A k )v = dn ( d k= f k(λ)a k )v = nonlinear eigenvector problem F (λ, v) =

5 a basic nonlinear eigenvalue problem Consider the simple scalar delay differential equation x (t) = x(t 1). Substituting the ansatz x(t) = e λt yields the nonlinear eigenvalue problem T (λ) = 1 + λe λ =. 32 (of infinitely many) eigenvalues of T for this scalar (n = 1) equation: Im λk eigenvalues determined by the Lambert-W function Re λ k See, e.g., [Michiels & Niculescu 27]

6 nonlinear eigenvalue problems: many resources Nonlinear eigenvalue problems have classical roots, but now form a fast-moving field with many excellent resources and new algorithms. Helpful surveys: Mehrmann & Voss, GAMM, [24] Voss, Handbook of Linear Algebra, [214] Güttel & Tisseur, Acta Numerica survey [217] Software: NLEVP test collection [Betcke, Higham, Mehrmann, Schröder, Tisseur 213] SLEPC contains NLEVP algorithm implementations [Roman et al.] Many algorithms based on Newton s method, rational approximation, linearization, contour integration, projection, etc. Incomplete list of contributors: Asakura, Bai, Betcke, Beyn, Effenberger, Güttel, Ikegami, Jarlebring, Kimura, Kressner, Leitart, Meerbergen, Michiels, Niculescu, Pérez, Sakurai, Tadano, Van Beeumen, Vandereycken, Voss, Yokota,.... Infinite dimensional nonlinear spectral problems are even more subtle: [Appell, De Pascale, Vignoli 24] give seven distinct definitions of the spectrum.

7 Rational Interpolation Algorithms for Nonlinear Eigenvalue Problems

8 rational interpolation of functions and systems Rational interpolation problem. Given points {z j } 2r j=1 C and data {f j f (z j )} 2r j=1, find a rational function R(z) = p(z)/q(z) of type (r 1, r 1) such that R(z j ) = f j.

9 rational interpolation of functions and systems Rational interpolation problem. Given points {z j } 2r j=1 C and data {f j f (z j )} 2r j=1, find a rational function R(z) = p(z)/q(z) of type (r 1, r 1) such that R(z j ) = f j. Given Lagrange basis functions l j (z) = r (z z k ) and nodal polynomial l(z) = k=1 k j r (z z k ), k=1 R(z) = p(z) q(z) = r β j l j (z) j=1 r w j l j (z) j=1 = r j=1 r j=1 β j l j (z) l(z) w j l j (z) l(z) = r β j z z j=1 j r w j z z j=1 j barycentric form

10 rational interpolation: barycentric perspective Lagrange basis: l j (z) = r (z z k ) k=1 k j R(z) = p(z) q(z) = r β j l j (z) j=1 r w j l j (z) j=1 = r β j z z j=1 j r w j z z j=1 j

11 rational interpolation: barycentric perspective Lagrange basis: l j (z) = r (z z k ) k=1 k j R(z) = p(z) q(z) = r β j l j (z) j=1 r w j l j (z) j=1 = r β j z z j=1 j r w j z z j=1 j Fix {β j = f j w j } r j=1 to interpolate at z 1,..., z r : R(z j ) = f j.

12 rational interpolation: barycentric perspective Lagrange basis: l j (z) = r (z z k ) k=1 k j R(z) = p(z) q(z) = r β j l j (z) j=1 r w j l j (z) j=1 = r β j z z j=1 j r w j z z j=1 j Fix {β j = f j w j } r j=1 to interpolate at z 1,..., z r : R(z j ) = f j. Determine w 1,..., w r to interpolate at z r+1,..., z 2r : R(z k ) = r f j w j z j=1 k z j r w j z j=1 k z j = f k = r j=1 f j w j z k z j = r j=1 f k w j z k z j

13 rational interpolation: barycentric perspective Fix {β j = f j w j } r j=1 to interpolate at z 1,..., z r : r(z j ) = f j. Determine w 1,..., w r to interpolate at z r+1,..., z 2r : R(z k ) = f k = r j=1 f j w j z k z j = r j=1 f k w j z k z j = r j=1 f j f k z j z k w j =.

14 rational interpolation: barycentric perspective Fix {β j = f j w j } r j=1 to interpolate at z 1,..., z r : r(z j ) = f j. Determine w 1,..., w r to interpolate at z r+1,..., z 2r : R(z k ) = f k = r j=1 f j w j z k z j = f 1 f r+1 z 1 z r+1 f 2 f r+1 z 2 z r+1 f 1 f r+2 z 1 z r+2 f 2 f r+2 z 2 z r+2. f 1 f 2r z 1 z 2r. f 2 f 2r z 2 z 2r r j=1 f k w j z k z j = f r f r+1 z r z r+1 f r f r+2 z r z r Loewner matrix, L f r f 2r z r z 2r w 1 w 2. w r r j=1 f j f k z j z k w j =. =.

15 rational interpolation: barycentric perspective Fix {β j = f j w j } r j=1 to interpolate at z 1,..., z r : r(z j ) = f j. Determine w 1,..., w r to interpolate at z r+1,..., z 2r : R(z k ) = f k = r j=1 f j w j z k z j = f 1 f r+1 z 1 z r+1 f 2 f r+1 z 2 z r+1 f 1 f r+2 z 1 z r+2 f 2 f r+2 z 2 z r+2. f 1 f 2r z 1 z 2r. f 2 f 2r z 2 z 2r r j=1 f k w j z k z j = f r f r+1 z r z r+1 f r f r+2 z r z r Loewner matrix, L f r f 2r z r z 2r w 1 w 2. w r r j=1 f j f k z j z k w j =. =. Barycentric rational interpolation algorithm [Antoulas & Anderson [1986] AAA (Adaptive Antoulas Anderson) Method [Nakatsukasa, Sète, Trefethen, 216]

16 rational interpolation: state space perspective The rational interpolant R(z) to f at z 1,..., z 2r can also be formulated in state-space form using Loewner matrix techniques. R(z) = c(l s zl) 1 b, where c = [f r+1,..., f 2r ], b = [f 1,..., f r ] T and z 1 f 1 z r+1 f r+1 z 1 z r+1. z 1 f 1 z 2r f 2r z 1 z 2r z r f r z r+1 f r+1 z r z r z r f r z 2r f 2r z r z 2r, f 1 f r+1 z 1 z r+1. f 1 f 2r z 1 z 2r f r f r+1 z r z r f r f 2r z r z 2r. shifted Loewner matrix, L s Loewner matrix, L State space formulation proposed by Mayo & Antoulas [27] Natural approach for handling tangential interpolation for vector data For details, applications, and extensions, see [Antoulas, Lefteriu, Ionita 217]

17 approach one: structure preserving rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Method: Reduce dimension of T(λ) but maintain the nonlinear structure. Smaller problem will be more amenable to dense nonlinear eigensolvers. Rational tangential interpolation of T(λ) 1 at r points, directions. Iteratively Corrected Rational Interpolation method

18 approach one: structure preserving rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Method: Reduce dimension of T(λ) but maintain the nonlinear structure. Smaller problem will be more amenable to dense nonlinear eigensolvers. Rational tangential interpolation of T(λ) 1 at r points, directions. Iteratively Corrected Rational Interpolation method Pick r interpolation points {z j } r j=1 and interpolation directions {w j } r j=1.

19 approach one: structure preserving rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Method: Reduce dimension of T(λ) but maintain the nonlinear structure. Smaller problem will be more amenable to dense nonlinear eigensolvers. Rational tangential interpolation of T(λ) 1 at r points, directions. Iteratively Corrected Rational Interpolation method Pick r interpolation points {z j } r j=1 and interpolation directions {w j } r j=1. Construct a basis for projection (cf. shift-invert Arnoldi ): U = orth([t(z 1) 1 w 1 T(z 2) 1 w 2 T(z r ) 1 w r ] C n r.

20 approach one: structure preserving rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Method: Reduce dimension of T(λ) but maintain the nonlinear structure. Smaller problem will be more amenable to dense nonlinear eigensolvers. Rational tangential interpolation of T(λ) 1 at r points, directions. Iteratively Corrected Rational Interpolation method Pick r interpolation points {z j } r j=1 and interpolation directions {w j } r j=1. Construct a basis for projection (cf. shift-invert Arnoldi ): U = orth([t(z 1) 1 w 1 T(z 2) 1 w 2 T(z r ) 1 w r ] C n r. Form the reduced-dimension nonlinear system: T r (λ) := U T(λ)U C r r.

21 approach one: structure preserving rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Method: Reduce dimension of T(λ) but maintain the nonlinear structure. Smaller problem will be more amenable to dense nonlinear eigensolvers. Rational tangential interpolation of T(λ) 1 at r points, directions. Iteratively Corrected Rational Interpolation method Pick r interpolation points {z j } r j=1 and interpolation directions {w j } r j=1. Construct a basis for projection (cf. shift-invert Arnoldi ): U = orth([t(z 1) 1 w 1 T(z 2) 1 w 2 T(z r ) 1 w r ] C n r. Form the reduced-dimension nonlinear system: T r (λ) := U T(λ)U C r r. Compute the spectrum of T r (λ) and use its eigenvalues and eigenvectors to update {z j } r j=1 and {w j } r j=1, and repeat.

22 approach one: structure preserving rational interpolation The choice of projection subspace Ran(U) delivers the key interpolation property. Interpolation Theorem. Provided z j σ(t) σ(t r ) for all j = 1,..., r, T(z j ) 1 w j = UT r (z j ) 1 U w j. Inspiration: model reduction for nonlinear systems w/coprime factorizations [Beattie & Gugercin 29]; iteration like dominant pole algorithm [Martins, Lima, Pinto 1996]; [Roomes & Martins 26], IRKA [Gugercin, Antoulas, Beattie 28].

23 approach one: structure preserving rational interpolation The choice of projection subspace Ran(U) delivers the key interpolation property. Interpolation Theorem. Provided z j σ(t) σ(t r ) for all j = 1,..., r, T(z j ) 1 w j = UT r (z j ) 1 U w j. Inspiration: model reduction for nonlinear systems w/coprime factorizations [Beattie & Gugercin 29]; iteration like dominant pole algorithm [Martins, Lima, Pinto 1996]; [Roomes & Martins 26], IRKA [Gugercin, Antoulas, Beattie 28]. Illustration. As for all orthogonal projection methods: T(λ) = f (λ) A + f 1(λ) A 1 + f 2(λ) A 2 T r (λ) = f (λ) U A U + f 1(λ) U A 1U + f 2(λ) U A 2U

24 approach one: structure preserving rational interpolation The choice of projection subspace Ran(U) delivers the key interpolation property. Interpolation Theorem. Provided z j σ(t) σ(t r ) for all j = 1,..., r, T(z j ) 1 w j = UT r (z j ) 1 U w j. Inspiration: model reduction for nonlinear systems w/coprime factorizations [Beattie & Gugercin 29]; iteration like dominant pole algorithm [Martins, Lima, Pinto 1996]; [Roomes & Martins 26], IRKA [Gugercin, Antoulas, Beattie 28]. Illustration. As for all orthogonal projection methods: T(λ) = f (λ) A + f 1(λ) A 1 + f 2(λ) A 2 T r (λ) = f (λ) U A U + f 1(λ) U A 1U + f 2(λ) U A 2U The nonlinear functions f j remain intact: the structure is preserved. The coefficients A j C n n are compressed to U A j U C r r. Contrast: [Lietaert, Pérez, Vandereycken, Meerbergen 218+] apply AAA approximation to f j (λ), leave coefficient matrices intact.

25 approach one: structure preserving rational interpolation Example 1. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} Eigenvalues of full T(λ) Interpolation points {z j } initial r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

26 approach one: structure preserving rational interpolation Example 1. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} Eigenvalues of full T(λ) Interpolation points {z j } Eigenvalues of reduced T r (λ) iteration r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

27 approach one: structure preserving rational interpolation Example 1. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} Eigenvalues of full T(λ) Interpolation points {z j } Eigenvalues of reduced T r (λ) iteration r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

28 approach one: structure preserving rational interpolation Example 1. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} Eigenvalues of full T(λ) Interpolation points {z j } Eigenvalues of reduced T r (λ) iteration r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

29 approach one: structure preserving rational interpolation Example 1. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} Eigenvalues of full T(λ) Interpolation points {z j } Eigenvalues of reduced T r (λ) iteration r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

30 approach one: structure preserving rational interpolation Example 1. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} zoom Eigenvalues of full T(λ) Eigenvalues of reduced T r (λ) at the final cycle Final interpolation points {z j } r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

31 approach one: structure preserving rational interpolation Example 1. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} convergence of r/2 rightmost (z j, w j ) pairs 2-2 T(zj)wj Eigenvalues of full T(λ) Eigenvalues of reduced T r (λ) at the final cycle Final interpolation points {z j } cycle r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

32 approach one: structure preserving rational interpolation Example 2. T(λ) = λi A e λ I, A is symmetric with n = 1; eigenvalues of A = { 1/n, 4/n,..., n} tight clustering of rightmost eigenvalues is a challenge zoom Eigenvalues of full T(λ) Eigenvalues of reduced T r (λ) at the final cycle Final interpolation points {z j } r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

33 approach one: structure preserving rational interpolation Example 2. T(λ) = λi A e λ I, A is symmetric with n = 1; eigenvalues of A = { 1/n, 4/n,..., n} convergence of r/2 rightmost (z j, w j ) pairs T(zj)wj Eigenvalues of full T(λ) Eigenvalues of reduced T r (λ) at the final cycle Final interpolation points {z j } cycle r = 16 used at each cycle (new points = real eigenvalues of T r (λ)) initial {z j } uniformly distributed on [ 1i, 1i], {w j } selected randomly

34 approach two: data-driven rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Obtain a small linear matrix pencil that interpolates the nonlinear eigenvalue problem. Smaller problem requires no further linearization. Method: Data-driven rational interpolation of T(λ) 1. Data-Driven Rational Interpolation Matrix Pencil method

35 approach two: data-driven rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Obtain a small linear matrix pencil that interpolates the nonlinear eigenvalue problem. Smaller problem requires no further linearization. Method: Data-driven rational interpolation of T(λ) 1. Data-Driven Rational Interpolation Matrix Pencil method Specify interpolation data: left points, directions: z 1,..., z r C, w 1,..., w r C n right points, directions: z r+1,..., z 2r C, w r+1,..., w 2r C n

36 approach two: data-driven rational interpolation Scenario: T(λ) C n n has large dimension n. Goal: Obtain a small linear matrix pencil that interpolates the nonlinear eigenvalue problem. Smaller problem requires no further linearization. Method: Data-driven rational interpolation of T(λ) 1. Data-Driven Rational Interpolation Matrix Pencil method Specify interpolation data: left points, directions: z 1,..., z r C, w 1,..., w r C n right points, directions: z r+1,..., z 2r C, w r+1,..., w 2r C n Construct T r (λ) 1 := C r (A r λe r ) 1 B r to tangentially interpolate T(λ) 1. Tangential Interpolation Theorem. Provided z j σ(t) σ(t r ), w T j T(z j ) 1 = w T j T r (z j ) 1, j = 1,..., r; T(z j ) 1 w j = T r (z j ) 1 w j, j = r + 1,..., 2r.

37 approach two: data-driven rational interpolation Given left points, directions: z 1,..., z r C, w 1,..., w r C n right points, directions: z r+1,..., z 2r C, w r+1,..., w 2r C n Define left interpolation data: f 1 = T(z 1) T w 1,..., f r = T(z r ) T w r right interpolation data: f r+1 = T(z r+1) 1 w r+1,..., f 2r = T(z 2r ) 1 w 2r

38 approach two: data-driven rational interpolation Given left points, directions: z 1,..., z r C, w 1,..., w r C n right points, directions: z r+1,..., z 2r C, w r+1,..., w 2r C n Define left interpolation data: f 1 = T(z 1) T w 1,..., f r = T(z r ) T w r right interpolation data: f r+1 = T(z r+1) 1 w r+1,..., f 2r = T(z 2r ) 1 w 2r Order-r (linear) model: T r (z) 1 = C r (A r ze r ) 1 B r

39 approach two: data-driven rational interpolation Given left points, directions: z 1,..., z r C, w 1,..., w r C n right points, directions: z r+1,..., z 2r C, w r+1,..., w 2r C n Define left interpolation data: f 1 = T(z 1) T w 1,..., f r = T(z r ) T w r right interpolation data: f r+1 = T(z r+1) 1 w r+1,..., f 2r = T(z 2r ) 1 w 2r Order-r (linear) model: T r (z) 1 = C r (A r ze r ) 1 B r C r = [ f r+1,..., f 2r ] A r = z 1 f1 T w r+1 z r+1 w1 T f r+1 z z 1 z r fr T w r+1 z r+1 wr T f r+1 r+1 z r z r z 1 f1 T w 2r z 2r w1 T f 2r z z 1 z r fr T w 2r z 2r wr T f 2r 2r z r z 2r E r = f1 T w r+1 w1 T f r+1 f z 1 z r T w r+1 w T r f r+1 r+1 z r z r f1 T w 2r w1 T f 2r f z 1 z r T w 2r wr T f 2r 2r z r z 2r B r = [ f 1,..., f r ] T shifted Loewner Loewner

40 approach two: data-driven rational interpolation Given left points, directions: z 1,..., z r C, w 1,..., w r C n right points, directions: z r+1,..., z 2r C, w r+1,..., w 2r C n Define left interpolation data: f 1 = T(z 1) T w 1,..., f r = T(z r ) T w r right interpolation data: f r+1 = T(z r+1) 1 w r+1,..., f 2r = T(z 2r ) 1 w 2r Rank-r (linear) model: T r (z) 1 = C r (A r ze r ) 1 B r (A r ze r) 1 B r T(z) 1 C r e.g., (A +f 1(z)A 1 +f 2(z)A 2) 1 linear matrix pencil

41 approach two: data-driven rational interpolation Example. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} zoom out Eigenvalues of full T(λ) Eigenvalues of reduced matrix pencil A r z E r r = 4 interpolation points used, uniform in interval [ 8i, 8i] Hermite interpolation variant that only uses r distinct interpolation points. interpolation directions from smallest singular values of T(z j ).

42 approach three: Loewner realization via contour integration Scenario: Seek all eigenvalues of T(λ) C n n in a prescribed region Ω of C. Goal: Use Keldysh s Theorem to isolate interesting part of T(λ) in Ω. Method: Contour integration of T(λ) against rational test functions. Loewner matrix will reveal number of eigenvalues in Ω. Theorem [Keldysh 1951]. Suppose T(z) has m eigenvalues λ 1,..., λ m (counting multiplicity) in the region Ω C, all semi-simple. Then T(z) 1 = V(zI Λ) 1 U + R(z), V = [v 1 v m], U = [u 1 u m], Λ = diag(λ 1,..., λ m), u j T (λ j )v j = 1; R(z) is analytic in Ω.

43 approach three: Loewner realization via contour integration Scenario: Seek all eigenvalues of T(λ) C n n in a prescribed region Ω of C. Goal: Use Keldysh s Theorem to isolate interesting part of T(λ) in Ω. Method: Contour integration of T(λ) against rational test functions. Loewner matrix will reveal number of eigenvalues in Ω. Theorem [Keldysh 1951]. Suppose T(z) has m eigenvalues λ 1,..., λ m (counting multiplicity) in the region Ω C, all semi-simple. Then T(z) 1 = V(zI Λ) 1 U + R(z), V = [v 1 v m], U = [u 1 u m], Λ = diag(λ 1,..., λ m), u j T (λ j )v j = 1; R(z) is analytic in Ω. T(z) 1 = H(z) + R(z) where H(z) := V(zI Λ) 1 U is a transfer function for a linear system.

44 approach three: Loewner approximation via contour integration Theorem [Keldysh 1951]. Suppose T(z) has m eigenvalues λ 1,..., λ m (counting multiplicity) in the region Ω C, all semi-simple. Then T(z) 1 = V(zI Λ) 1 U + R(z), V = [v 1 v m], U = [U 1 U m], Λ = diag(λ 1,..., λ m), U j T (λ j )v j = 1; R(z) is analytic in Ω. (zi Λ) 1 U T(z) 1 = V + R(z) H(z) := V(zI Λ) 1 U n n linear system, order m: m poles in Ω nonlinear system, but nice in Ω

45 approach three: Loewner realization via contour integration T(z) 1 = H(z) + R(z) where H(z) : V(zI Λ) 1 U is a transfer function for a linear system. A family of algorithms use the fact that, by the Cauchy integral formula, 1 f (z)t(z) 1 dz = Vf (Λ)U ; 2πi Ω see [Asakura, Sakurai, Tadano, Ikegami, Kimura 29], [Beyn 212], [Yokota & Sakurai 213], etc., building upon contour integral eigensolvers for matrix pencils [Sakurai & Sugiura 23], [Polizzi 29], etc. These algorithms use f (z) = z k for k =, 1,... to produce Hankel matrix pencils.

46 approach three: Loewner realization via contour integration T(z) 1 = H(z) + R(z) where H(z) : V(zI Λ) 1 U is a transfer function for a linear system. A family of algorithms use the fact that, by the Cauchy integral formula, 1 f (z)t(z) 1 dz = Vf (Λ)U ; 2πi Ω see [Asakura, Sakurai, Tadano, Ikegami, Kimura 29], [Beyn 212], [Yokota & Sakurai 213], etc., building upon contour integral eigensolvers for matrix pencils [Sakurai & Sugiura 23], [Polizzi 29], etc. These algorithms use f (z) = z k for k =, 1,... to produce Hankel matrix pencils. Key observation: If we use f (z) = 1/(z j z) for z j exterior to Ω, we obtain 1 1 2πi z j z T(z) 1 dz = V(z j I Λ) 1 U = H(z j ). Ω Contour integrals yield measurements of the linear system with the desired eigenvalues.

47 approach three: Loewner realization via contour integration Minimal Realization via Rational Contour Integrals for m eigenvalues Let r m, and select interpolation points and directions: left points, directions: z 1,..., z r C \ Ω, w 1,..., w r C n right points, directions: z r+1,..., z 2r C \ Ω, w r+1,..., w 2r C n

48 approach three: Loewner realization via contour integration Minimal Realization via Rational Contour Integrals for m eigenvalues Let r m, and select interpolation points and directions: left points, directions: z 1,..., z r C \ Ω, w 1,..., w r C n right points, directions: z r+1,..., z 2r C \ Ω, w r+1,..., w 2r C n Use contour integrals to compute the left and right interpolation data: left interpolation data: f 1 = H(z 1) T w 1,..., f r = H(z r ) T w r right interpolation data: f r+1 = H(z r+1)w r+1,..., f 2r = H(z 2r )w 2r H(z j )w j = 1 1 2πi Ω z j z T(z) 1 w j dz.

49 approach three: Loewner realization via contour integration Minimal Realization via Rational Contour Integrals for m eigenvalues Let r m, and select interpolation points and directions: left points, directions: z 1,..., z r C \ Ω, w 1,..., w r C n right points, directions: z r+1,..., z 2r C \ Ω, w r+1,..., w 2r C n Use contour integrals to compute the left and right interpolation data: left interpolation data: f 1 = H(z 1) T w 1,..., f r = H(z r ) T w r right interpolation data: f r+1 = H(z r+1)w r+1,..., f 2r = H(z 2r )w 2r H(z j )w j = 1 1 2πi Ω z j z T(z) 1 w j dz. Construct Loewner and shifted Loewner matrices from this data, just as in the Data-Driven Rational Interpolation method: C r = [ f r+1,..., f 2r ] A r = shifted Loewner matrix B r = [ f 1,..., f r ] T E r = Loewner matrix

50 approach three: Loewner realization via contour integration Minimal Realization via Rational Contour Integrals for m eigenvalues Let r m, and select interpolation points and directions: left points, directions: z 1,..., z r C \ Ω, w 1,..., w r C n right points, directions: z r+1,..., z 2r C \ Ω, w r+1,..., w 2r C n Use contour integrals to compute the left and right interpolation data: left interpolation data: f 1 = H(z 1) T w 1,..., f r = H(z r ) T w r right interpolation data: f r+1 = H(z r+1)w r+1,..., f 2r = H(z 2r )w 2r H(z j )w j = 1 1 2πi Ω z j z T(z) 1 w j dz. Construct Loewner and shifted Loewner matrices from this data, just as in the Data-Driven Rational Interpolation method: C r = [ f r+1,..., f 2r ] A r = shifted Loewner matrix B r = [ f 1,..., f r ] T E r = Loewner matrix If r = m, then V(zI Λ) 1 U = C r (A r z E r ) 1 B r : compute eigenvalues! If r > m, use SVD truncation / minimum realization techniques to reduce dimension; cf. [Mayo & Antoulas 27].

51 approach three: Loewner realization via contour integration Example. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n} zoom, N = Eigenvalues of full T(λ) 2 interpolation points in 2 + [ 6i, 6i] Eigenvalues of minimal (m = 4) matrix pencil Contour of integration (circle) Trapezoid rule uses N = 25, 5, 1, and 2 interpolation points

52 approach three: Loewner realization via contour integration Example. T(λ) = λi A e λ I, where A is symmetric with n = 1; eigenvalues of A = { 1, 2,..., n}. jth singular value of L j eigenvalue error, λj λ (N) j N 4 eigenvalues in Ω rank(l) = 4 Cf. [Beyn 212], [Güttel & Tisseur 217] for f (z) = z k. For rank detection for Loewner matrices, see [Hokanson 218+].

53 Transient Dynamics for Dynamical Systems with Delays a case study of pseudospectral analysis

54 introduction to transient dynamics We often care about eigenvalues because we seek insight about dynamics.

55 introduction to transient dynamics Start with the simple scalar system We often care about eigenvalues because we seek insight about dynamics. x (t) = αx(t), with solution x(t) = e tα x(). If Re α <, then x(t) monotonically as t. 1.5 x(t) t

56 introduction to transient dynamics We often care about eigenvalues because we seek insight about dynamics. Now consider the n-dimensional system x (t) = Ax(t) with solution x(t) = e ta x(). If Re λ < for all λ σ(a), then x(t) asymptotically as t, 2 x(t) A = [ 1 ] t

57 introduction to transient dynamics We often care about eigenvalues because we seek insight about dynamics. Now consider the n-dimensional system x (t) = Ax(t) with solution x(t) = e ta x(). If Re λ < for all λ σ(a), then x(t) asymptotically as t, but it is possible that x(t ) x() for some t (, ). 2 x(t) A = [ 1 ] t

58 why transients matter Often the linear dynamical system x (t) = Ax(t) arises from linear stability analysis for a fixed point of a nonlinear system y (t) = F(y(t), t). For example, y (t) = Ay(t) y(t) y(t) x(t) A = [ 1 ] In this example, linear transient growth feeds the nonlinearity. Such behavior can provide a mechanism for transition to turbulence in fluid flows; see, e.g., [Butler & Farrell 1992], [Trefethen et al. 1993]. t

59 detecting the potential for transient growth One can draw insight about transient growth from the numerical range (field of values) and ε-pseudospectra of A: σ ε(a) = {z C : (zi A) 1 > 1/ε} = {z C : z σ(a + E) for some E C n n with E < ε} For upper and lower bounds on x(t), see [Trefethen & E. 25], e.g., sup e ta Re z If σ ε(a) extends sup t z σ ε(a) ε. more than ε across the imaginary axis, e ta grows transiently.

60 detecting the potential for transient growth One can draw insight about transient growth from the numerical range (field of values) and ε-pseudospectra of A: σ ε(a) = {z C : (zi A) 1 > 1/ε} = {z C : z σ(a + E) for some E C n n with E < ε} For upper and lower bounds on x(t), see [Trefethen & E. 25], e.g., sup e ta Re z If σ ε(a) extends sup t z σ ε(a) ε. more than ε across the imaginary axis, e ta grows transiently σ ε (A) e ta lower bound on sup t> e ta from σ ε(a) with ε = 1/ log 1 ε t Pseudospectra can guarantee that some x() induce transient growth.

61 two ways to look at pseudospectra Two equivalent definitions give two distinct perspectives. perturbed eigenvalues σ ε(a) = {z C : z σ(a + E) for some E C n n with E < ε} norms of resolvents σ ε(a) = {z C : (zi A) 1 > 1/ε}

62 two ways to look at pseudospectra Two equivalent definitions give two distinct perspectives. perturbed eigenvalues σ ε(a) = {z C : z σ(a + E) for some E C n n with E < ε} norms of resolvents σ ε(a) = {z C : (zi A) 1 > 1/ε} σ ε(a) contains the eigenvalues of all matrices with distance ε of A. Ideal for assessing asymptotic stability of uncertain systems: Is some matrix near A unstable? Why consider all E C n n? Structured pseudospectra further restrict E (real, Toeplitz, etc.). [Hinrichsen & Pritchard], [Karow], [Rump]

63 two ways to look at pseudospectra Two equivalent definitions give two distinct perspectives. perturbed eigenvalues σ ε(a) = {z C : z σ(a + E) for some E C n n with E < ε} σ ε(a) contains the eigenvalues of all matrices with distance ε of A. Ideal for assessing asymptotic stability of uncertain systems: Is some matrix near A unstable? Why consider all E C n n? Structured pseudospectra further restrict E (real, Toeplitz, etc.). [Hinrichsen & Pritchard], [Karow], [Rump] norms of resolvents σ ε(a) = {z C : (zi A) 1 > 1/ε} σ ε(a) is bounded by 1/ε level sets of the resolvent norm. Ideal for assessing transient behavior of stable systems: e ta > 1 or A k > 1? Rooted in semigroup theory: based on the solution operator for the dynamical system; structure of A plays no role.

64 two ways to look at pseudospectra Two equivalent definitions give two distinct perspectives. perturbed eigenvalues σ ε(a) = {z C : z σ(a + E) for some E C n n with E < ε} σ ε(a) contains the eigenvalues of all matrices with distance ε of A. Ideal for assessing asymptotic stability of uncertain systems: Is some matrix near A unstable? Why consider all E C n n? Structured pseudospectra further restrict E (real, Toeplitz, etc.). [Hinrichsen & Pritchard], [Karow], [Rump] norms of resolvents σ ε(a) = {z C : (zi A) 1 > 1/ε} σ ε(a) is bounded by 1/ε level sets of the resolvent norm. Ideal for assessing transient behavior of stable systems: e ta > 1 or A k > 1? Rooted in semigroup theory: based on the solution operator for the dynamical system; structure of A plays no role. These perspective match for x (t) = Ax(t), but not for more complicated systems.

65 scalar delay equations and the nonlinear eigenvalue problem We shall apply these ideas to explore the potential for transient growth in solutions to stable delay differential equations. Solutions of scalar systems x (t) = αx(t) behave monotonically: x(t) = e t Re α x(). What about scalar delay equations? x (t) = αx(t) + βx(t 1) Using the techniques seen earlier, we associate this system with the NLEVP (λ α)e λ = β, with infinitely many eigenvalues given by branches of the Lambert-W function: λ k = α + W k (βe α ).

66 scalar delay equations and the nonlinear eigenvalue problem We shall apply these ideas to explore the potential for transient growth in solutions to stable delay differential equations. Solutions of scalar systems x (t) = αx(t) behave monotonically: x(t) = e t Re α x(). What about scalar delay equations? {λ k } for { α = 3/4 β = x (t) = αx(t) + βx(t 1) = λ k = α + W k (βe α ).

67 scalar delay equations and the nonlinear eigenvalue problem We shall apply these ideas to explore the potential for transient growth in solutions to stable delay differential equations. Solutions of scalar systems x (t) = αx(t) behave monotonically: x(t) = e t Re α x(). What about scalar delay equations? Re λ k < for all k = asymptotic stability {λ k } for { α = 3/4 β = x (t) = αx(t) + βx(t 1) = λ k = α + W k (βe α ).

68 stability chart for two-parameter delay equation Conventional eigenvalue-based stability analysis reveals the (α, β) combinations that yield asymptotically stable solutions unstable (α, β) pairs.5 β stable (α, β) pairs α Such stability charts are standard tools for studying stability of parameter-dependent delay systems.

69 pseudospectra for nonlinear eigenvalue problems Green & Wagenknecht [26] and Michiels, Green, Wagenknecht, & Niculescu [26] define pseudospectra for nonlinear eigenvalue problems, and apply them to delay differential equations. See [Cullum, Ruehli 21], [Wagenknecht, Michiels, Green 28], [Bindel, Hood 213]. Consider the nonlinear eigenvalue problem T(λ)v = with T(λ) = m f j (λ) A j. For p, q [1, ] and weights w 1,..., w m (, ], define the norm w 1 E 1 q (E 1,..., E m) p,q =.. w m E m q p Given this way of measuring a perturbation to T(λ), [MGWN 26] define σ ε(t) = { z C : z σ ( m j=1 ) f j (λ) (A j + E j ) for some j=1 } E 1,..., E m C n n with (E 1,..., E m) p,q < ε.

70 pseudospectra for the scalar delay equation x (t) = αx(t) + βx(t 1) T (λ) = λ α βe λ MGWN ε-pseudospectra for α = 3 and β = 1, 4 with perturbation norm given by q [1, ] and p =, and w 1 = w 2 = 1.

71 pseudospectra for the scalar delay equation x (t) = x(t) + x(t 1) T (λ) = λ + 1 T (λ) = λ + 1 e λ 4 log log log log log log 1 1 MGWN ε-pseudospectra with p = : structure affects pseudospectra.

72 the solution operator To better understand transient behavior, just integrate the differential equation: x (t) = αx(t) + βx(t 1) history: x(t 1) = u(t) for t [, 1].

73 the solution operator To better understand transient behavior, just integrate the differential equation: x (t) = αx(t) + βx(t 1) history: x(t 1) = u(t) for t [, 1]. Integrate to get, for t [, 1], x (t) = αx(t) + βu(t) x(t) = e t α x() + β = e t α u(1) + β t t e (t s)α u(s) ds e (t s)α u(s) ds.

74 the solution operator To better understand transient behavior, just integrate the differential equation: x (t) = αx(t) + βx(t 1) history: x(t 1) = u(t) for t [, 1]. Integrate to get, for t [, 1], x (t) = αx(t) + βu(t) x(t) = e t α x() + β = e t α u(1) + β t t e (t s)α u(s) ds e (t s)α u(s) ds. This operation maps the history u to the solution x for t [, 1]: u C([, 1]) x C([, 1]).

75 the solution operator Define the solution operator K : C[, 1] C[, 1] via x(t) = (Ku)(t) = e t α u(1) + β t e (t s)α u(s) ds, t [, 1].

76 the solution operator Define the solution operator K : C[, 1] C[, 1] via x(t) = (Ku)(t) = e t α u(1) + β t e (t s)α u(s) ds, t [, 1]. x m = x(t) t [m 1,m] define: x () := u t [ 1, ] to advance t by 1 unit, apply K: x (1) := Kx () t [, 1] to advance t by 2 units, apply K 2 : x (2) := Kx (1) = K 2 x () t [1, 2]. to advance t by m units, apply K m : x (m) := Kx (m 1) = K m x () t [m 1, m]

77 the solution operator Define the solution operator K : C[, 1] C[, 1] via x(t) = (Ku)(t) = e t α u(1) + β t e (t s)α u(s) ds, t [, 1]. x m = x(t) t [m 1,m] define: x () := u t [ 1, ] to advance t by 1 unit, apply K: x (1) := Kx () t [, 1] to advance t by 2 units, apply K 2 : x (2) := Kx (1) = K 2 x () t [1, 2]. to advance t by m units, apply K m : x (m) := Kx (m 1) = K m x () t [m 1, m] View the delay system as a discrete-time dynamical system over 1-unit time intervals: x (m) = K m x ().

78 discretizing the solution operator We discretize the solution operator using a Chebyshev pseudospectral method based on [Trefethen 2]; see [Bueler 27], [Jarlebring 28]. N tj x(t j ) x j := e t j α u + βw j,k u k, w j,k := e (t j s)α l k (s) ds k= x e t α u w, w,1 w,n u x 1 e t 1α u 1 w 1, w 1,1 w 1,N u 1 = β x N e t N α }{{} E N (α) u N w N, w N,1 w N,N }{{} W N (α) u N

79 discretizing the solution operator We discretize the solution operator using a Chebyshev pseudospectral method based on [Trefethen 2]; see [Bueler 27], [Jarlebring 28]. N tj x(t j ) x j := e t j α u + βw j,k u k, w j,k := e (t j s)α l k (s) ds k= x e t α u w, w,1 w,n u x 1 e t 1α u 1 w 1, w 1,1 w 1,N u 1 = β x N e t N α }{{} E N (α) u N w N, w N,1 w N,N }{{} W N (α) u N K N := E N (α) + β W N (α) x (1) := K N u x (m) := K m N u

80 approaches to transient analysis of delay equations Jacob Stroh [26], in a master s thesis advised by Ed Bueler, computes L 2 -pseudospectra of Chebyshev discretizations of the compact solution operator and considers nonnormality as a function of a time-varying coefficient in the delay term: our approach follows closely. Green & Wagenknecht [26], in their paper about perturbation-based pseudospectra for delay equations, describe computing the pseudospectra of the generator for the solution semigroup as a way of gauging transient behavior; for relevant semigroup theory, see, e.g., [Engel & Nagel 2]. Hood & Bindel [216+] apply Laplace transform/pseudospectral techniques to the solution operator for delay differential equations for upper/lower bounds on transient behavior. See also the Lyapunov approach to analyzing transient behavior in the 25 Ph.D. thesis of Elmar Plischke. Solution operator approach converts a finite dimensional nonlinear problem into an infinite dimensional linear problem, akin to the infinite Arnoldi algorithm [Jarlebring, Meerbergen, Michiels 21, 212, 214].

81 convergence of the eigenvalues of the solution operator To study convergence, consider α =, β = 1: x (t) = x(t 1). µ (N) j : the jth largest magnitude eigenvalue of K N e λ j : λ j is the jth rightmost eigenvalue of the NLEVP eigenvalue error µ (N) j e λj j N We generally use N = 64 for our computations throughout what follows.

82 nonconvergence of the L 2 pseudospectra of the solution operator Eigenvalues converge, but the L 2 [, 1] pseudospectra of K N do not: the departure from normality increases with N! N = N = N = N = 256

83 the problem with the L 2 norm Problem: The L 2 (, 1) norm does not measure transient growth of x(t). One can easily find u(x) such that u L 2 [,1] 1 but x L 2 [,1] 1. Let α =, β = 1: x (t) = x(t 1) = x(t) = u(1) t u(s) ds. 1 x(t) -1 u(t) = exp(1(t 1)) initial condition t 1 x(t) -1 u(t) = exp(1(t 1)) initial condition t

84 the problem with the L 2 norm Problem: The L 2 (, 1) norm does not measure transient growth of x(t). One can easily find u(x) such that u L 2 [,1] 1 but x L 2 [,1] 1. Let α =, β = 1: x (t) = x(t 1) = x(t) = u(1) t u(s) ds. 15 x (m) L 2 [,1] u L 2 [,1] m

85 pseudospectra and transient growth of matrix powers Since we care about the largest value x(t) can take, we should really study x (m) L, and thus the ε-pseudospectrum σ ε(k N ) defined using the -norm: σ ε(k N ) := {z C : (zi K N ) 1 > 1/ε} := {z C : z σ(k N + E) for some E C n n with E < ε}.

86 pseudospectra and transient growth of matrix powers Since we care about the largest value x(t) can take, we should really study x (m) L, and thus the ε-pseudospectrum σ ε(k N ) defined using the -norm: σ ε(k N ) := {z C : (zi K N ) 1 > 1/ε} := {z C : z σ(k N + E) for some E C n n with E < ε}. Even in Banach spaces, pseudospectra give lower bounds on transient growth; see, e.g., [Trefethen & E., 25]. sup K m z 1 sup m z σ ε(k) ε If σ ε(k) extends more than ε outside the unit disk, K m grows transiently. Limitations: [Greenbaum & Trefethen 1994], [Ransford et al. 27, 29, 211]

87 stability versus solution operator norm x (t) = αx(t) + βx(t 1) unstable (α, β) pairs.5 β stable (α, β) pairs α Stable choices of the (α, β) parameters

88 stability versus solution operator norm x (t) = αx(t) + βx(t 1) ρ(k) = 1.5 β α Level sets: ρ(k) =.1,.2,..., 1.

89 stability versus solution operator norm x (t) = αx(t) + βx(t 1) 2 K = K = β α Level sets: K =.5, 1.,..., 4.5

90 stability versus solution operator norm x (t) = αx(t) + βx(t 1) β α Superimposed level sets for ρ(k) and K

91 stability versus solution operator norm x (t) = αx(t) + βx(t 1) β sweet spot for transient growth α Superimposed level sets for ρ(k) and K

92 solution matrix pseudospectra ( -norm) x (t) = αx(t) + βx(t 1) α = 1 β = 1 ρ(k) = 1 K =

93 solution matrix pseudospectra ( -norm) x (t) = αx(t) + βx(t 1) α = β =.99 ρ(k) =.99 K =

94 solution matrix pseudospectra ( -norm) x (t) = αx(t) + βx(t 1) α = β =.9 ρ(k) =.9 K =

95 solution matrix pseudospectra ( -norm) x (t) = αx(t) + βx(t 1) α = β =.99 ρ(k) =.99 K = α = β =.9 ρ(k) =.9 K =

96 solution operator: transient growth x (t) = αx(t) + βx(t 1) x(t) ρ(k) =.9 x(t) initial condition t ρ(k) = initial condition t

97 solution operator: transient growth x (t) = αx(t) + βx(t 1) x(t) ρ(k) =.9 ρ(k) =.99 ρ(k) = t As α 1 and β 1, solutions exhibit arbitrary transient growth, but slowly.

98 can scalar equations exhibit stronger transients? Is faster transient growth possible in a scalar equation if we allow multiple synchronized delays? x (t) = c x(t) + c 1 x(t 1) + c 2 x(t 2) + + c d x(t d).

99 can scalar equations exhibit stronger transients? Is faster transient growth possible in a scalar equation if we allow multiple synchronized delays? x (t) = c x(t) + c 1 x(t 1) + c 2 x(t 2) + + c d x(t d). Key: Look for solutions of the form x(t) = t d e λt.

100 can scalar equations exhibit stronger transients? Is faster transient growth possible in a scalar equation if we allow multiple synchronized delays? x (t) = c x(t) + c 1 x(t 1) + c 2 x(t 2) + + c d x(t d). Key: Look for solutions of the form x(t) = t d e λt. One can show that x(t) = t d e λt is a solution if and only if c, c 1,..., c d solve the Vandermonde linear system d 1 4 d d d d c e λ c 1 e 2λ c 2... e dλ c d = λ 1....

101 commensurate delays can give much larger pseudospectra x (t) = c x(t) + c 1x(t 1) + + c d x(t d) 1 d = 1 d = 2 d = ρ(k) =.9 ρ(k) =.9 ρ(k) = d = 4 d = ρ(k) =.9 ρ(k) =

102 commensurate delays can induce strong transients x (t) = c x(t) + c 1 x(t 1) + c 2 x(t 2) + + c d x(t d) Initial data: x(t) = 1 + 2e 1t for t d = c =.8946 c 1 =.9 15 d = c = c 1 = 1.8 c 2 = d = c = c 1 = 2.7 c 2 = c 3 =.243 d = c = c 1 = 3.96 c 2 = 2.43 c 3 =.972 c 4 =.164

103 commensurate delays can induce strong transients x (t) = c x(t) + c 1x(t 1) + + c d x(t d) 1 6 x(t) d = 5 d = t With commensurate delays, solutions to scalar equations can exhibit significant transient growth very quickly in time.

104 summary rational interpolation for nlevps Rational / Loewner techniques motivated by algorithms from model reduction Structure Preserving Rational Interpolation: iteratively improve projection subspaces via interpolation points and directions. Data-Driven Rational Interpolation Matrix Pencils: reduce nonlinear problem to linear matrix pencil with tangential interpolation property. Minimal Realization via Rational Contour Integrals: isolates a transfer function for a linear system, recover via Loewner minimal realization techniques. transients for delay equations Solutions to scalar delay equations can exhibit strong transient growth. Finite dimensional nonlinear problem infinite dimensional linear problem Pseudospectral theory applies to the linear problem, but the choice of norm is important. Chebyshev collocation keeps the discretization matrix size small. Adding commensurate delays enables a faster rate of initial transient growth.