A fast and well-conditioned spectral method: The US method

Transcription

1 A fast and well-conditioned spectral method: The US method Alex Townsend University of Oxford Leslie Fox Prize, 24th of June 23 Partially based on: S. Olver & T., A fast and well-conditioned spectral method, to appear in SIAM Review, (23). Alex Townsend, University of Oxford 24th of June 23 /26

2 The problem Problem: Solve linear ordinary differential equations (ODEs) a N (x) d N u dx N + + a (x) du dx + a (x)u = f (x), x [, ] with K linear boundary conditions (Dirichlet, Neumann, u = c, etc.) Assumptions: a,..., a N, f are continuous with bounded variation. The leading term a N (x) is non-zero on the interval. The solution exists and is unique. Main philosophy: Replace a,..., a N, f by polynomial approximations, and solve for a polynomial approximation for u. The spectral method finds the coefficients in a Chebyshev series in O(m 2 n) operations: n u(x) α k T k (x), k= T k (x) = cos(k cos x). Alex Townsend, University of Oxford 24th of June 23 /26

3 Overview 6 4 degree(u) = 55 Conventional wisdom A comparison First-order ODEs Higher-order ODEs Stability and convergence Fast linear algebra PDEs Conclusion y u(x) x x Alex Townsend, University of Oxford 24th of June 23 2/26

4 Quotes about spectral methods It is known that matrices generated by spectral methods can be dense and ill-conditioned. [Chen 25] The idea behind spectral methods is to take this process to the limit, at least in principle, and work with a differentiation formula of infinite order and infinite bandwidth, i.e., a dense matrix. [Trefethen 2] The best advice is still this: use pseudospectral (collocation) methods instead of spectral (coefficient), and use Fourier series and Chebyshev polynomials in preference to more exotic functions. [Boyd 23] The Ultraspherical spectral method (US method) can result in almost banded, well-conditioned linear systems. Alex Townsend, University of Oxford 24th of June 23 3/26

5 Chebyshev spectral methods u (x) + cos(x)u(x) =, Collocation Coefficients Condition number u(±) =. US method Density Matrix structure Condition number Problem sizes Operation count Collocation Dense O(n2N ) n 5, O(n3 ) Alex Townsend, University of Oxford Coefficient Banded from below O(n2N ) n 5, O(mn2 ) 24th of June 23 US method Banded + rank-k O(n2(D ) ) n 7, O(m2 n) 4/26

6 First-order differential equations Differentiation Operator Multiplication Operator u (x) + a(x)u(x) = f(x) (D + SM[a]) u = Sf D : T C () M[a] : T T S : T C () β Hypergeometric functions Jacobi with w(x)=( x) α (+x) β T L Differentiation C () Conversion Jacobi Ultraspherical Hypergeometric functions α Alex Townsend, University of Oxford 24th of June 23 5/26

7 First-order operators Differentiation operator: D : T C (), [DLMF] { dt k dx = kc () k, k, D =, k =, Conversion operator: S : T C (), [DLMF] ( ) 2 C () k C () k 2, k 2, T k = 2 C (), k =, C (), k =, Multiplication operator: M[a] : T T, [DLMF] T j T k = 2 ( T j k + T j+k ), j, k S = Alex Townsend, University of Oxford 24th of June 23 6/26

8 First-order multiplication operator T j T k = 2 T j k + 2 T j+k 2a a a 2 a M[a] = a 2a a a a 2 a 2a a.. + a a 2 a 3 a a 2 a 3 a 4 a 5... a 3 a 2 a 2a.. a 3 a 4 a 5 a }{{}}{{} Toeplitz Hankel + rank- Multiplication is not a dense operator in finite precision. It is m-banded: a(x) = a k T k (x) = k= m ã k T k (x) + O(ɛ), k= where ã k are aliased Chebyshev coefficients. Alex Townsend, University of Oxford 24th of June 23 7/26

9 Imposing boundary conditions Dirichlet: u( ) = c: B = (,,,,..., ), T k ( ) = ( ) k. Neumann: u () = c: B = (,, 4, 9,..., ), T k () = k 2. Other conditions: u = c, u() = c, or u() + u (π/6) = c. Imposing boundary conditions by boundary bordering: u ( ) (x)+a(x)u(x) = f, B = Bu = c, D + SM[a] ( c f ). The finite section can be done exactly by projection P n = (I n, ): ( P n DP T n BP T n + P n SP T n+mp n+m M[a]P T n ) = ( c P n f ). Alex Townsend, University of Oxford 24th of June 23 8/26

10 First-order examples Example u (x) + x 3 u(x) = sin(2,x 2 ), u( ) =. The exact solution is u(x) = e x4 4 ( x e t4 4 ) sin(2,t 2 )dt. u(x) degree(u) = 239 time = 5.5s x N = ultraop(@(x,u) ); N.lbc = ; u = N \ f; Adaptively selects the discretisation size. Forms a chebfun object [Chebfun V4.2]. ũ u =.5 5. Alex Townsend, University of Oxford 24th of June 23 9/26

11 First-order examples Example 2 u (x) + u(x) =, u( ) =. + 5,x 2 The exact solution with a = 5, is ( u(x) = exp tan ( ax) + tan ( ) a). a.5 5 u(x) degree(u) = x Absolute error 5 Old chebop Preconditioned collocation US method x Alex Townsend, University of Oxford 24th of June 23 /26

12 Higher-order linear ODEs Higher-order diff operators: D λ : T C (λ), [DLMF] λ times dc (λ) { k 2λC (λ+) {}}{ = k, k, D λ = 2 λ (λ )! λ dx, k =, λ Higher-order conversion operators: S λ : C (λ) C (λ+), [DLMF] ( ) λ C (λ+) λ+k k C (λ+) k 2, k 2, λ λ+2 k = λ λ+ C (λ+) λ, k =, S λ = λ λ+ λ+3. C (λ+)., k =,..... C (λ) Multiplication operators: M λ [a] : C λ C λ, M [a] : C () C (), M [a] = Toeplitz + Hankel. β Jacobi with w(x)=( x) α (+x) β Differentiation Conversion Jacobi Ultraspherical α T L Alex Townsend, University of Oxford 24th of June 23 /26 C () C (2)

13 Construction of ultraspherical multiplication operators M λ [a] : C (λ) C (λ) represents multiplication with a(x) in C (λ). If a(x) = m j= a j C (λ) j (x), then it can be shown that M λ [a] is m-banded with (M λ [a]) j,k = k s=max(,k j) where (to prevent overflow issues) we have cs λ (j, k) = j + k + λ 2s s j + k + λ s s t= a 2s+j k c λ s (k, 2s + j k), j, k, t= j s λ + t + t j s 2λ + j + k 2s + t λ + j + k 2s + t t= t= λ + t + t k s + + t k s + λ + t. For efficiency, we use a recurence relation to generate c λ s (j, k). Alex Townsend, University of Oxford 24th of June 23 2/26

14 A high-order example a N (x) d N u dx N + an (x) d N u dx N + + a (x)u = f (x), Bu = c, ( B ) M N [a N ]D N + S N M N [a N ]D N + + S N S M [a ] Example 3: High-order ODE ( = u () (x) + cosh(x)u (8) (x) + cos(x)u (2) (x) + x 2 u(x) = u(±) =, u (±) =, u (k) (±) =, k = 2, 3, 4. c S N S f ).5 u(x) u n+ u n ( ) (ũ(x) + ũ( x)) 2 2 = x n Alex Townsend, University of Oxford 24th of June 23 3/26

15 Airy equation and regularity preserving Example 4 Consider Airy s equation for ɛ >, ) ( ) ɛu (x) xu(x) =, u( ) = Ai ( 3 /ɛ 3, u() = Ai /ɛ. Condition Number ɛ = 9 ɛ = 4 ɛ = n x 4 Relative error Relative error in derivative at x=/2 Ai () Ai (2) Ai () n The US method can accurately compute high derivatives of functions. For a different approach, see [Bornemann 2]. Alex Townsend, University of Oxford 24th of June 23 4/26

16 Stability when K = N Assumption: Number of boundary conditions = Differential order. Definition: Higher-order l 2 λ -norms For λ N, l 2 λ C is the Banach space with norm u 2 l = u 2 k 2 ( + k) 2λ <. λ k= Right diagonal preconditioner: R = 2 N (N )! I N N N+.... Alex Townsend, University of Oxford 24th of June 23 5/26

17 Stability when K = N Recall the notation: Solve Lu = f, with Bu = c, B : l 2 D C K, bounded, D. Theorem: Condition number is bounded with n ( B Let : l L) λ+ l2 λ be an invertible operator for some λ D. Then, as n, A n R n l 2 λ = O(), (A n R n ) l 2 λ = O(), where R n = P T n RP n, A n = P T n ( B L) P n. Proof uses integral reformulation ideas: R integrates the ODE to form a Fredholm operator, i.e., ( B R = I + K, K = compact. L) Alex Townsend, University of Oxford 24th of June 23 6/26

18 Convergence with K = N Theorem: Computed solution converges in high-order norms ( ) B Suppose f l 2 λ N+ for some λ D, and that : l L 2 λ+ l2 λ is an invertible operator. Define ( ) u n = A c n P n, S N S f then u P n u n l 2 λ+ C u P n P n u l 2 λ+. The constant, C, does not depend on n. You don t always need the complex plane to compute high derivatives. Relative error 5 5 Relative error in derivative at x=/2 Ai () Ai (2) Ai () n Alex Townsend, University of Oxford 24th of June 23 7/26

19 Fast linear algebra: The QTP factorisation Original matrix: After left Givens: After right Givens: Apply a partial factorisation on the left, followed by a partial factorisation on the right. Factorisation: A n = QTP in O(m 2 n) operations. No fill-in: The matrix T contains no more non-zeros than A n. For solving A n x = b: The matrix Q can be immediately applied to b. The matrix P can be stored using Demmel s trick [Demmel 997]. UMFPACK by Davis is faster for n 5, but has observed complexity of, roughly, O(n.25 ) with a very small constant. Alex Townsend, University of Oxford 24th of June 23 8/26

20 Constant coefficient PDEs Consider the constant coefficient PDE N N j i+j u a ij i x j = f (x, y), y a j= i= ij R defined on [a, b] [c, d] with linear boundary conditions satisfying continuity conditions. We discretise as a generalised Sylvester matrix equation k σ j A j XBj T = F, A j R n n, B j R n2 n2, j= where A j and B j are US discretisations, for example, A j = ( ) Alex Townsend, University of Oxford 24th of June 23 9/26

21 Determining the rank of a PDE operator Given a PDE with constant coefficients of the form, L = We can rewrite this as N N j j= i= a ij i+j i x j y, A = (a ij). L = D(y) T AD(x), D(x) = / x.. N / x N. Hence, if A = UΣV T is the truncated SVD of A with Σ R k k then L = D(y) T ( UΣV T ) D(x) = ( D(y) T U ) Σ ( V T D(x) ). This low rank representation for L tells us how to discretize and solve the PDE. Alex Townsend, University of Oxford 24th of June 23 2/26

22 Matrix equation solvers Rank-: A XB T = F. Solve A Y = F, then B X T = Y T. Rank-2: A XB T + A 2XB2 T = F. Generalised Sylvester solver [Jonsson & Kågström, 992]. Rank-k, k 3: Solve n n 2 n n 2 system with QTP factorization. O(n3 n 2) n & n 2 O(n 3 + n3 2 ) O(n n 2 ) Boundary conditions are imposed by carefully removing degrees of freedom in the matrix equation. Automatically forms and solves subproblems, if possible. Corner singularities can be partly resolved by domain subdivision. Alex Townsend, University of Oxford 24th of June 23 2/26

23 Examples Example 5: Helmholtz equation uxx + uyy + 2ω 2 u =, u(±, y ) = f (±, y ), u(x, ±) = f (x, ±), where f (x, y ) = cos(ωx) cos(ωy ). ω = 5 N = chebop2(@(u)... ); N.lbc=f(-,:); u=n\; y.5 Adaptively selects the discretisation size. Forms a chebfun2 object [T. & Trefethen 23]..5.5 x.5 For ω = 2, k u uk2 = For ω = 5 solve takes.27s for n = n2 = 26, for ω = 2 solve takes 32.s for n = n2 = 224. Alex Townsend, University of Oxford 24th of June 23 22/26

24 Conclusions The US method results in almost banded well-conditioned matrices. It requires O(m 2 n) operations to solve an ODE. It can be used as a preconditioner for the collocation method. The US method converges and is numerically stable. It can be extended to a spectral method for solving PDEs on rectangular domains. Alex Townsend, University of Oxford 24th of June 23 23/26

25 Historical example [Orszag 97] Orr-Sommerfeld equation (u 2u + u) 2iu i( x 2 )(u u) = λ(u u), u(±) = u (±) =. Eigenvalues of the Orr Sommerfeld operator imag real Alex Townsend, University of Oxford 24th of June 23 24/26

26 The end Thank you More information: S. Olver & T., A fast and well-conditioned spectral method, to appear in SIAM Review, (23). Alex Townsend, University of Oxford 24th of June 23 25/26

27 References F. Bornemann, Accuracy and stability of computing high-order derivatives of analytic functions by Cauchy integrals, Found. Comput. Math., (2), pp. 63. I. Jonsson & Kågström, Recursive blocked algorithms for solving triangular matrix equations Part II: Two-sided and generalized Sylvester and Lyapunov equations, ACM Trans. Math. Softw., 28 (22), pp S. Olver & T., A fast and well-conditioned spectral method, to appear in SIAM Review, (23). S. Orszag, Accurate solution of the Orr-Sommerfeld stability equation, J. Fluid Mech., 5 (97), pp T. & L. N. Trefethen, An extension of Chebfun to two dimensions, likely to appear in SISC, (23). L. N. Trefethen et al., The Chebfun software system, version 4.2, 2. Alex Townsend, University of Oxford 24th of June 23 26/26

28 Chebyshev polynomials An underappreciated fact: Every Chebyshev series can be rewritten as a palindromic Laurent series. The degree k Chebyshev polynomial (of the first kind) is defined by T k (x) = cos ( k cos x ) = 2 ( z k + z k), x [, ], where z = e ikθ and cos θ = x. T 25 (x).5.5 T 25 (θ) = cos(25θ).5.5 x θ Every Chebyshev series can be written as a palindromic Laurent series: N α k T k (x) = 2 k= N α k z k + α + 2 k= N α k z k, z = e ikθ. k= Alex Townsend, University of Oxford 24th of June 23 27/26

29 Demmel s trick for storing Givens rotations A Givens rotation zeros out one entry of a matrix and we can store the rotation information there. Givens {}}{ p Let s = sin θ and c = cos θ, where θ is rotation angle, then { s sign(c), s < c, p = sign(s) c, s c. To restore (up to a 8 degrees): If p <, then s = p and c = s 2, otherwise c = /p and s = c 2. Alex Townsend, University of Oxford 24th of June 23 28/26